Identifying and prioritising midwifery care process metrics and indicators: a Delphi survey and stakeholder consensus process

Midwives engage daily in numerous healthcare interventions where their knowledge, clinical expertise and professional judgement guide and influence their decision-making to ensure high quality, safe care delivery. Knowing however what quality midwifery care is, and how to measure it has always been a challenge, both in Ireland and internationally [1]. Many quality improvement approaches in maternity tend to focus on care outcomes, such as mortality and morbidity, length of hospital stay, neonatal or maternal admissions to intensive/special care and readmission rates. For example, the top ten most frequently used outcomes in 32 newly published 2011 Cochrane systematic reviews of intrapartum interventions were; admission to neonatal intensive care unit, maternal satisfaction, Apgar scores < 7 at 5 min, perinatal mortality, breastfeeding rates, caesarean section, instrumental birth, pain, adverse events and infection [2]. Measuring outcomes, which may be used to reflect the quality of care, is an important healthcare indicator. To determine however the quality of midwifery care, and in particular midwives contribution to the safety of women and their infants, requires midwives to be able to clearly articulate and measure what it is that they do; that is, midwifery clinical care processes.

Measuring care processes enables healthcare providers to have insight to the quality of care delivery and to establish improvement action-plans that will ultimately lead to better outcomes for maternity services users. In the absence of this, confirming associations between care processes and short- or long-term outcomes for pregnant/postpartum women can be challenging. This is because care processes extend beyond usual care outcomes, and hold implications for how care is provided and evaluated, as well as measured [3]. Because midwives represent the largest group of healthcare professionals in the care of women and babies [4] it is important that their work is made visible and that their significant contribution to maternity outcomes is recognised.

In 2016, the Office of Nursing and Midwifery Services Directorate in Ireland commissioned a national research study to establish the important dimensions of nursing and midwifery care processes that should be measured [5]. These dimensions aimed to reflect care delivery that is sensitive to the influences of nurses and midwives aligned to evidenced-based clinical practice guidelines and standards developed for and within the context of nursing and midwifery care in Ireland. The culmination of this work has resulted in a suite of seven Quality Care Metrics (QCM) reports that outline these metrics and associated indicators in the healthcare areas of Midwifery, Children’s Community/Public Health, Acute, Older People, Mental Health and Intellectual Disability [6]. The objective of this paper is to report on the development and prioritisation of a national suite of QCM, and their associated indicators, for midwifery practice in Ireland. Additional file 1 presents the midwifery work-stream working group members.

The study comprised of four discrete, yet complimentary, phases. In phase 1 a systematic review to bring together available and relevant literature on reported quality care process metrics and associated indicators across all seven of the work-stream areas (Midwifery, Children’s Community/Public Health, Acute, Older People, Mental Health and Intellectual Disability) to inform the development of a suite of process sensitive metrics and their associated indicators. Metrics and indicators identified in the systematic review were subsequently tagged against their relevant work-stream area and used to develop work-stream specific surveys for use in phases 2 and 3. Phases 2 and 3 consisted of two by two-round e-Delphi surveys to identify and prioritise a suite of metrics (phase 2) and their indicators (phase 3) for use in measuring the quality of midwifery care processes in Ireland. An e-Delphi survey is a research method that involves a series of questionnaires, called ‘rounds’, administered electronically to a panel of relevant stakeholders on a topic under investigation so as to gather their opinions. The results of each round are presented to participants in subsequent rounds, with participants asked to provide their opinion again based on the knowledge of the collective group results from the previous round. It has been described as an optimal design for facilitating consensus-building on a topic under investigation [7]. The fourth and final phase involved a face-to-face consensus meeting with midwives (n = 19) to review the findings from the Delphi surveys and to agree on the final suite of QCM, and their respective indicators, for midwifery care.

A total of 441 midwives expressed an interest, by email, in participating in phase 2 (prioritising metrics), of which 263 participated in round 1 of the metric survey. Just over one third of participants were staff midwives (34.6%, n = 91) and almost one quarter were clinical midwife managers (grade 2) (24.3%, n = 64). A large proportion of participants identified their grade of midwifery as ‘other’ (12.2%, n = 32). Of these, most were either clinical skills facilitators (18.8%, n = 6) or clinical placement coordinators (18.8%, n = 6) (Table 1). Of the 263 respondents who completed round 1, 69.1% (n = 183) completed Round 2. Over half of the respondents to round 2 were either staff midwives (26.8%, n = 49) or clinical midwife managers (grade 2) (29.5%, n = 54). A large proportion of the participants in round 2 also identified their grade of midwifery as ‘other’, that is, clinical skills facilitators or clinical placement coordinators (27.5%, n = 38) (Table 1).

Participants rated 21 of the 22 metrics included in phase 2, round 1, as important for inclusion in the suite. In addition, nine metrics were identified newly by participants. These, plus the 21 metrics, were carried forward to round 2. In round 2, participants rated all 30 metrics as important for inclusion in the suite. These 30 metrics were discussed in detail by the midwifery working group where three (Women’s Experience, Irish-Maternity Early Warning Score (I-MEWS) and Invasive Medical Devices) were identified as having a separate process either underway or planned for which indicators were or would be developed. For this reason, these three metrics were not included in phase 3 of the project within which indicators to measure adherence to the metrics were prioritised. In addition, eight metrics were judged to overlap with other metrics and were removed. The remaining 19 metrics were carried forward to phase 3 and later to the face to face consensus meeting (phase 4) along with participants’ suggestions for where metrics may overlap.

A total of 217 midwives participated in the phase 3, round 1 Delphi survey (prioritising indicators). Just over half of the respondents were staff midwives (30.0%, n = 65) and clinical midwife managers (grade 2) (25.4%, n = 55) (Table 3). Of the 217 midwives who completed round 1, 69.6% (n = 151) of these completed round 2. Just over one quarter of respondents to this final round were staff midwives (25.89%, n = 39) and one-fifth were clinical midwife managers (grade 2) (19.9%, n = 30). A large proportion of participants (19.2%, n = 29) identified their grade of midwifery as ‘other’, which consisted largely of clinical placement coordinators (20.7%, n = 6) (Table 2).

Of the 109 indicators included in phase 3, participants, in round 1, rated all as important for inclusion in the midwifery metrics suite. In addition, 1 indicator was newly identified by participants. In round 2, participants rated the 110 indicators as important for inclusion in the suite. Following detailed review and discussion by the working group, seven indicators were judged to lack clarity, were potentially ambiguous and were removed. The remaining 103 indicators and the associated 19 metrics, were carried forward to the face to face consensus meeting.

This study describes a strategy of identifying and prioritising a suite of 18 metrics and 93 associated indicators to measure midwifery care processes. Measuring the quality of the process of midwifery care is complex [59]. The metrics and indicators presented here offer an important understanding of the interplay between care delivery, measurement and care outcomes and how maternity system improvement through the actions and interventions of midwives might be achieved. The metrics and indicators are not designed necessarily to offer an exhaustive list, nor do we consider that they should be used solely in isolation of contextual issues, including variation in national/regional models of care. Organisations should aim to achieve consensus on a set of measures including structural, process and outcome data to guide the delivery of high quality safe care provision across the maternity care continuum, from antenatal through to the postpartum period. The current set of QCM and indicators were developed specifically with the Irish maternity care system in mind, and we accept that care systems can vary internationally, as well as regionally. In Ireland, for example, a national survey in 2014 indicated that 69% of 2820 surveyed women would like a model of midwifery care (e.g. midwifery-led care in hospital, home birth, birth centres) available to them, however, only 20% were able to avail of this type of care [60]. This is largely reflective of the type of maternity care offered in Ireland, with 19 maternity hospitals across the country, and the availability of only two midwifery-led units alongside consultant-led hospital units, 14 self-employed community midwives, and no stand-alone birth centres. Acknowledging this, we believe the QCM reported here can be used or adapted for use in other countries and settings, while recognising that care processes might be context specific [59].

This research process and final set of midwifery QCM and indicators were identified and prioritised using a methodologically robust and rigorous process. Importantly, the widespread engagement in the project by midwives of all grades and geographical areas nationally, via the work stream groups and project officers, has ensured that there is a real sense of ownership of the metrics and indicators from midwives across settings. This, in turn, has ensured relevance and will enhance direct transferability to clinical midwifery practice. We recognise, however, some limitations to this work. For example, our sample size falls short of the a priori sample size of 318 (using a 95% confidence level and a confidence interval of ±5); however, our sample size at round 2 of both metrics and indicators surveys achieved a 95% confidence level with confidence intervals of ±7 and ± 8 respectively. Staff midwife grades were under represented somewhat in the Delphi surveys despite extensive efforts to hear their views. While a maternity service user was a member of the project Steering Group (SG), we also acknowledge that the voice of pregnant and postpartum women and their families is largely absent. This decision was made at SG level, because the project SG felt the focus should be on midwives.

Although not the aim of our work, the systematic review in phase 1 of the study identified a dearth of evidence on studies evaluating the effectiveness of metrics and indicators on quality of care and thus, a need for research assessing such effectiveness is recommended. Follow-up on this initial work is intended via a rigorous evaluation of the impact of the new suite of metrics on midwifery care processes. Designs that control, insofar as is possible for confounding variables such as interrupted time series designs will be considered, and determined prior to implementation so that opportunities for baseline assessments are not lost.

Knowing what midwives do, and how they do it, is a fundamental component to achieving high quality maternity care. The result of this study (i.e. the suite of metrics and indicators) offers a basis for embedding the concept of measurement for improvement in midwifery practice in order to assure the delivery of high quality, safe maternity care. Use of the suite of QCM will also facilitate measurement of and accountability in care provision, and will assist, ultimately, in achieving the goal of improved maternal, fetal and neonatal outcomes.