Standardized outcome measures for pregnancy and childbirth, an ICHOM proposal

ICHOM convened a Working Group composed of two consumer representatives and 19 international experts in various fields of perinatal and neonatal care, research and patient advocacy. Within the realm of feasibility, Working Group members were selected to provide balanced expertise across geographies and clinical specialties, as well as representation from obstetric registries and outcomes measurement initiatives (Table 1). The activities of the Working Group were coordinated by a Project Team consisting of a Working Group lead (Franx), a Project Lead (Wissig), a Research Fellow (Nijagal), and the ICHOM Vice President of Research & Development (Stowell).

The measure set was developed using a modified Delphi method [14]. Between May 2015 and May 2016, the Working Group convened for nine teleconferences. Excluding the launch and final meetings, each teleconference had a pre-determined, specific goal such as establishing the scope of the measure set, defining the patient population, selecting outcomes and case-mix domains, identifying appropriate definitions and/or measures for each domain, and determining when each measure would be assessed during the pregnancy and postpartum course. Based on the goal, the Project Team reviewed relevant literature and current practices prior to the teleconference and presented this information, along with a specific proposal, during the teleconference for group discussion. Detailed minutes of these discussions were distributed following each teleconference to Working Group members, who then voted on each item of the Project Team’s proposal via an online survey. Items required a 70% agreement among survey respondents to be finalized into the measure set. Survey items with less than 70% majority were either excluded from the set or revised by the project team and re-presented for discussion and voting at the next teleconference.

Multiple information sources were sought to support the consideration of outcome domains to be included. In addition to reviewing outcomes included in regional perinatal health registries and quality indicator sets, a comprehensive literature review was performed using search terms focused on quality outcomes or indicators, birth experience and health-related quality of life (HRQoL). This resulted in a comprehensive list of both clinical and woman-centered outcomes. [Additional file 1]. A serial guided discussion among five pregnant and postpartum women was also conducted to identify additional outcomes that had not emerged from the literature search. Participants in this focus group were asked to reflect on their most significant experiences during the pregnancy, birth and postpartum periods as a mechanism to explore what participants’ felt were their most important goals of care. The group represented a variety of ages, parities, phases in the care cycle (prenatal vs. postpartum), clinical experiences (routine vs. complicated), and nationalities. We recognize that this did not provide a representative sample of pregnant and postpartum women globally; however, our aim was to gather further information to support decision-making and guide prioritization of outcome domains by the Working Group.

The comprehensive list of potential outcome domains was presented to the Working Group for discussion during a teleconference meeting. Working Group members were then asked to score each potential outcome on the GRADE scale via electronic survey [15]. Outcome domains thought to be “critical” (scored between 7 and 9) by at least 70% of the respondents were included in the set. Those scored as “low importance” (between 1 and 3) by at least 70% of respondents were excluded. The remaining domains were modified and re-presented for a second round of voting. Domains meeting neither the inclusion nor exclusion criterion after a second round of voting were discussed again by the Working Group and then presented for a final binary vote.

A similar protocol was followed to define appropriate measures for each domain, and to select the case-mix factors included in the set. Prior to teleconferences, the project team reviewed the literature to identify potential measures for each domain, and to compile a comprehensive list of demographic, social, and clinical factors associated with the selected outcomes. The final outcome measures and case-mix factors were then finalized through the process of Working Group discussion via teleconference, followed by voting via electronic survey.

To determine when and how each outcome measure and case-mix factor would be assessed during the pregnancy to postpartum continuum, the Project Team used the same process: current practices were researched, options discussed with the Working Group during teleconferences, and electronic surveys were administered for voting.

To ensure robust consumer input in the development of the measure set, we solicited feedback from pregnant and postpartum women around the world via an anonymous online survey. Quorum Review IRB issued a written determination of exemption for the ICHOM Patient Advisory Group in Pregnancy and Childbirth. A link to the survey was distributed within Working Group members’ networks via social media, with no inclusion or exclusion criteria for participation. The survey presented, in lay terms, the outcome domains voted in for inclusion by the Working Group. Respondents were asked to score included domains according to their importance on the GRADE scale and were given an opportunity at the end of the survey to suggest any missing outcomes. Survey responses and suggestions were presented to the Working Group to inform their conclusion on the generalizability of the consumer advisory group discussion themes.

To also allow for input from healthcare professional stakeholders outside of the formal Working Group, a 4-week open review period was held prior to the last Working Group teleconference. The Project Team identified key stakeholders representing provider organizations, payers, consumer advocacies, and other individuals expressing interest in the measure set via the ICHOM website. Each was sent an overview of the set with links to the full detail Reference Guide and a feedback survey. The results of this survey were presented to the Working Group during the final teleconference call.

Response rates for the seven post-teleconference surveys present to Working Group members were 82, 82, 73, 73, 77, 77 and 73% respectively. Group size fluctuated due to the late addition of some members and occasional unavailability of others. All members received call minutes and were kept abreast of the Working Group’s progress. For post-teleconference surveys that involved two rounds of voting, the response rate for the second round is presented here.

The measure set covers key outcomes of care for all women and their infants from the first prenatal visit through six months postpartum. The endpoint was selected as a pragmatic compromise: the Working Group recognized that while important outcomes may not emerge until later than 6 months after birth [16], the response rates for patient questionnaires decreases over time and, therefore, a much later endpoint may not be feasible [17, 18]. Pregnancies with pre- or postnatally diagnosed significant congenital anomalies are excluded from measurement.

All five participants had one or more children; one was pregnant at the time of the discussion and four were postpartum.

Seven major themes emerged from the discussion.

These themes were presented to the working group during the second teleconference call.

Outcome domains and definitions/measures included in the set are presented in Table 2, along with the percentage of responding Working Group members who agreed with the inclusion of the domain. Domains and measures for which there was significant discussion within the Working Group are discussed below.

A number of patient characteristics and risk factors are known to influence the outcomes presented above. To ensure fair comparisons across providers with diverse patient populations, the Working Group identified and defined key case-mix factors to include in the set. Factors selected for inclusion were considered to have a strong and independent effect on the outcomes included in the set, and to be practical for collection in an international setting. All case-mix factors and definitions are presented in Table 3, along with the percent of responding Working Group members who agreed upon their inclusion. The outcome of preterm birth also allows for stratification of other maternal and infant outcomes that may be impacted by gestational age at delivery.

The timeline for measurement was constructed based on clinical relevance and feasibility (Fig. 1). First, timeframes for measuring each outcome were identified based on clinical appropriateness. Next, recommended care schedules from several countries were analyzed to identify common time points at which women engage with maternity care. Tying patient-reported data collection to common clinic appointments allows collection to happen within the clinic and use of the data within clinical care. The 6-month postpartum data collection point is beyond the time frame of standard maternity care internationally and requires data to be collected from women via mail or electronic platforms.

Minimizing the length of patient surveys was a priority to reduce survey burden on women. Recognizing that not all women desire to breastfeed, the BSES-SF was made an optional measure to identify those who may benefit from additional support in the hospital or early postpartum period [35]. The Working Group also recommended a hierarchical question design when assessing outcomes affecting only a subset of women. For example, questions about the nature and frequency of urinary or fecal incontinence are burdensome for women without incontinence. Therefore, a single ICHOM-defined incontinence screening question is presented to all women with only those reporting symptoms going on to complete validated PROMs assessing symptom severity. Similarly, the Edinburgh Postnatal Depression Scale (EPDS) is included as an optional follow-up measure for those who screen positive on the shorter PHQ-2 [36‐38]. Both measures have been validated for the pregnancy and the postnatal periods: the PHQ-2 is a practical and sensitive measure to detect perinatal depression, while the EPDS provides higher specificity [36]. Of note, while the Working Group advocated for postpartum depression screening by all maternity care providers, they emphasized that a response protocol must be in place to identify and treat individiuals who screen positive in a timely manner.

A total of 105 consumer validation surveys and 17 complete responses to the open review feedback survey were received from across all continents except South America. Responses were generally positive. For the open review feedback, a median score of 4 (“agree”) on a 5-point scale from “strongly disagree” to “strongly agree” was obtained for statements about the scope of the measure set, the appropriateness of the included measures, and its ease of implementation. 94% of respondents reported that they would recommend implementation of the measure set to their colleagues. Specific survey comments were presented to the Working Group for discussion but resulted in no changes to the measure set.

Our work represents a unique contribution to health systems and providers seeking to improve perinatal care delivery. To our knowledge, this is the first internationally developed set of perinatal measures that: (a) focuses on outcomes that matter to women, rather than processes of care, (b) includes PROMs, and (c) includes a set of case-mix factors to facilitate outcome comparisons. By involving consumers in our work process and focusing on the goal of overall wellbeing of mothers and infants, we identified common pregnancy outcomes and experiences that may be overlooked by health care professionals, but have a major impact on physical and psychological wellbeing.

There were a number of limitations in this work. First, ICHOM aims to create measure sets that are appropriate across cultures, applicable in diverse health care settings, and practical to implement. However, for low-income, low-resource countries with high rates of mortality and high levels of morbidity, measurement of comprehensive perinatal outcomes may be less compelling and too burdensome at this time. Accordingly, although the Working Group represented a diverse range of middle- and high-income countries, representation from low-income countries was limited.

Second, it was challenging for the group to identify and agree on validated measures for each outcome domain. As discussed above, the Working Group unanimously agreed that severe maternal morbidity was an important outcome to include in the measures set. However, agreeing on the best measures to capture this outcome proved challenging. The “life-threatening condition” approach used by WHO was considered difficult to implement as it requires clinical report and may not be representative of severe morbidity in high-income countries [22]. In contrast, the approach used by the Center for Disease Control (CDC) [25] of using administrative data to track the incidence of 25 adverse maternal outcomes was considered too broad and cumbersome. In addition, the incidence of each of these adverse outcomes is typically quite low in advanced economies, limiting the use of this data for quality improvement [52]. As a compromise, the Working Group selected a handful of proxy measures (ICU admission, length of stay, pregnancy-related readmission, and blood transfusion) that are easily measured and have been shown to capture cases of significant adverse maternal outcomes [23, 53]. While these proxy measures may be considered processes rather than outcomes, each was considered an important outcome from the perspective of women as they each represent a delay in return to normal activity (prolonged facility stay), cause separation from their infant (ICU admission and postpartum readmission), or introduce new risk (blood transfusion).

Similar factors influenced the selection of other new or non-validated measures for inclusion in the set. The Working Group recognizes that these measures must be tested and validated over time, and ICHOM is committed to supporting this process. Implementors of measures in this set are encouraged to inform ICHOM of their work and share their experiences. A Steering Committee comprised of ICHOM Working Group members has been assembled to guide the continued maintenance and refinement of the set based on input from these early adopters. As measures are refined and implementation expands, ICHOM will work with implementors to validate measures as necessary.

Finally, the practicality of measurement and the burden of data collection in the clinical setting is always an important consideration. Although data abstracted from administrative records may have limited accuracy, capturing clinical data directly from providers is often prohibitively burdensome [54, 55]. Therefore, the measure set consists of a small number of administratively captured data points and relies heavily upon patient-reported data. This approach has proven successful in a variety of data collection efforts around the world [56‐58], although capturing patient-reported outcomes remains a challenge, particularly in low- and middle- income countries. Along with helping a number of care delivery organizations with implementation of the measure set, ICHOM has partnered with PharmAccess Foundation to explore the possibility of using mobile phone technology to enable routine collection of patient-reported data in Kenya. While the number of electronic options for collecting such data continues to expand, distribution of paper surveys within the clinic remains a low-cost option [59].