Validating the WHO maternal near miss tool: comparing high- and low-resource settings

One of the Millennium Development Goals was to reduce global maternal mortality in 2015 by three quarters as compared to the level of 1990 [1]. In the summer of 2015, the United Nations reported an estimated 45% decline (using data up to 2013), indicating that this target would not be fully met. In the meantime, new Sustainable Development Goals have been set, including the reduction of the maternal mortality ratio below 70 per 100.000 live births by 2030 [2]. Assessment of pregnant women with severe maternal outcome (SMO), comprised of maternal near miss (MNM) and maternal death (MD), may contribute to accelerating this morbidity and mortality reduction [3].

The World Health Organisation (WHO) has defined a MNM as a ‘woman who nearly died but survived a complication that occurred during pregnancy, childbirth or within 42 days of termination of pregnancy’ [3, 4]. WHO proposes a ‘MNM approach’ to monitor and improve quality of obstetric care using a tool that classifies women according to several (potentially) life-threatening conditions (Table 1) [4]. The classification is based on three different types of criteria: disease-, intervention- and organ dysfunction-based. If any of the organ dysfunction-based criteria are met, the MNM approach defines that case as ‘life-threatening’, and therefore MNM [5].

According to WHO, uniformity of this MNM classification should make it possible to compare the quality of obstetric care between different settings in different countries, which would be useful in improving health care delivery. However, in some low-resource settings application of the WHO MNM tool showed underreporting of life-threatening maternal morbidity. This may be due to lack of blood for transfusion, absence of laboratory diagnostics and poor clinical monitoring, which are all needed to identify MNM [6‐8].

In a nationwide cohort, we previously found that also in the Netherlands, a high-resource setting, organ dysfunction-based criteria failed to identify almost 60% of women with severe acute maternal morbidities as MNM [9]. If these women, who were not detected as having had ‘life-threatening’ conditions, had attended obstetric care in low-resource settings the majority would likely have died.

Our previous studies have highlighted difficulties in finding universal criteria to identify MNM and raise questions about the applicability of the MNM tool in general, and its focus on organ dysfunction-based criteria in particular [6‐9]. The objective of this study is to analyse merged data of one high- and two low-resource settings where this tool was applied and test whether the tool may be suitable for comparing SMO between these settings.

In this current study, we used merged data available from SMO databases collected in the Netherlands, Tanzania and Malawi. Data for the Netherlands were extracted from a two-year nationwide cohort study (the LEMMoN-study), for Tanzania from a two-year cross-sectional study at Haydom Lutheran Hospital and for Malawi from a two-year study of maternal morbidity and mortality at Thyolo District Hospital (the ‘4 M-study’). A general description of the three study populations can be found in Table 1. Details and outcomes for these three cohorts have been published previously [7‐10].

Women with SMO were included according to definitions established by the original studies (Table 2). We reassessed all cases in these three cohorts using the WHO MNM tool which defines MNM based on three different types of criteria: disease-, intervention- and organ dysfunction-based. Fourteen cases (0.4%) of the Dutch cohort were excluded due to insufficient data for application. All other 2538 SMO patients were assessed without the need for supplementation of any marker [9]. For the low-resource settings, identification of SMO did not only depend on relatively advanced laboratory tests, but could also happen on the basis of supplemented clinical markers as recommended by WHO [5].

Data from the three studies were collected into a single database containing the following variables: age (<20, 20-35 and >35 years), parity (0, 1 and ≥2), units of blood given (0, 1, 2, 3, 4 and ≥5), duration of hospital stay, maternal mortality, and classification according to the three WHO MNM tool criteria groups (disease-, intervention- and organ dysfunction-based). If women had multiple conditions or interventions they were included into more than one criteria group, with each included criterion titled a separate ‘event’. Case fatality rates (CFR) were calculated for the corresponding populations.

All parameters were compared between each country’s population and those women who sustained life-threatening conditions as per WHO definition. Outcomes for the three countries were analysed individually and compared for differences. Finally, the life-threatening group was corrected by including every Tanzanian and Malawian woman (where giving five or more units is an exception even in life-threatening haemorrhage [11]) who received one unit or more of blood for transfusion. Maintaining five units of blood as an organ dysfunction criterion would imply that in settings where the availability of blood products is severely limited, fewer MNM cases are included.

Data were analysed using chi-square tests for categorical data and independent sample t-tests for numerical data. Statistical analysis was performed using SPSS statistics, version 20.0 (SPSS, Chicago, IL). All three initial studies had ethical approval and for present study anonymous data were used.

A total of 3172 women were analysed: 2538 (80.0%) from the Netherlands, 248 (7.8%) from Tanzania, and 386 (12.2%) from Malawi. General characteristics of all three populations are shown in Table 3. All parameters significantly differed between the three countries.

After assessment with the WHO MNM tool, out of the 2538 Dutch women, 2308 (90.9%) fulfilled one or more disease-based criteria, 2116 (83.4%) any intervention-based criterion and 1024 (40.3%) any organ dysfunction-based criterion. In Tanzania there were 123 (49.6%) women fulfilling disease-based, 231 (85.9%) intervention-based, and 103 (41.5%) organ dysfunction-based criteria. For Malawi these numbers were 336 (87.0%), 175 (45.3%), and 84 (21.8%), respectively. The detection in the combined study population of 3172 women was 2767 (87.2%) women for disease-based, 2504 (78.9%) for intervention-based, and 1211 (38.2%) for organ dysfunction-based criteria. Only this final group sustained ‘life-threatening conditions’ according to WHO methodology. The CFRs were 48/2538 (1.9%) for the Netherlands, 32/248 (12.9%) for Tanzania and 46/386 (11.9%) for Malawi. Of these maternal deaths, 17 (35%) women in the Netherlands and 15 (33%) women in Malawi could not be identified as having had a ‘life-threatening’ condition. In Tanzania, all maternal deaths could be defined.

For the total population, analysis of the events detected by the WHO MNM tool subcategories is shown in Table 4. Postpartum haemorrhage (PPH) is the most commonly detected event among the disease-based criteria. Pre-eclampsia follows as an important second in the Netherlands, whereas in Tanzania and Malawi sepsis is more prominent. Giving blood products is the most frequent intervention and laparotomies (other than caesarean section) are more frequently performed in Malawi and Tanzania compared to the Netherlands. For the organ dysfunction-based criteria, coagulation or haematological dysfunction is the major reason for inclusion in the Netherlands, whereas in the low-resource settings this is cardiovascular dysfunction. Between countries all subcategories differed significantly except for the numbers of ruptured uterus (disease-based), admissions to ICU (intervention-based), and women who presented with renal dysfunction or ended up having hysterectomy (organ dysfunction-based).

Among women with life-threatening conditions (as defined by the organ dysfunction-based criteria, Table 5), PPH is the most common event for inclusion in the Netherlands and Tanzania. In Malawi PPH, eclampsia, infection, and uterine rupture are almost equally represented. Eclampsia is significantly more common in both low-resource settings. Giving blood products is the commonest intervention-based criterion in the Netherlands and Malawi. In Tanzania this is ICU admission.

After correction for any blood transfusion in the low-resource settings the life-threatening group changed (Table 5). First, the MNM tool now identified 1458 (46.0%) women with organ dysfunction, instead of 1205 (38.2%). In addition, blood transfusion became a more frequent inclusion criterion in the low-resource settings as compared to the Dutch setting, and ‘coagulation or hematologic dysfunction’ was now equally represented in each setting. When including any blood transfusion, the position of PPH as major contributor to severe acute maternal morbidity becomes more prominent in Tanzania and Malawi (36.4% and 24.4% raised to 53.2% and 42.6%).

The WHO MNM tool inclusions and general characteristics of women with life-threatening conditions (before and after correction for blood transfusion) can be seen in Tables 6 and 7. In comparison with the total study population (Table 3) higher CFRs are seen among women with life-threatening conditions, and among women in low-resource settings.

Our results indicate that the WHO MNM tool, in its current form, is not useful for comparison between different resource settings. Detection differs between high- and low-income countries and organ dysfunction-based criteria detect only 38.2% of all women with SMO as defined by the three cohort studies.

A recent study by Menezes et al. states that the WHO criteria perform well [12]. In this study, conducted in two Brazilian reference hospitals, 77 out of 1196 (6.4%) women were identified as having life-threatening conditions based on the WHO MNM tool, compared to 33.8% and 80.2% by using Waterstone’s or other literature-based criteria respectively. However, the authors do not clarify why the other 1119 (93.6%) women did not sustain MNM conditions or why these pregnant women did not ‘nearly die, but survived’ (according to WHO MNM definition). The reason for this omission appears that the current WHO criteria are mistakenly seen as the ‘gold standard’ for evaluation of severe maternal morbidity.

The underestimation of severe maternal outcome when applying the WHO MNM tool in its current form remains an important issue. Overall, disease-based criteria show the highest detection of SMO (87.2%) in each type of setting. An explanation for the low detection rate (49.6%) in the Tanzanian population could be the local SMO criteria used in that study. For example, this led to fewer women with PPH (according to the WHO MNM definition of blood loss above one liter) in this cohort, as PPH as such was no separate inclusion criterion in the Tanzanian cohort (in contrast with Malawi) and women were only included if they had received blood transfusion. The intervention-based criteria detected 78.9% of all SMO cases. An explanation for the low detection (45.3%) in the Malawian population is the absence of interventional radiology and an ICU. Both disease-based and intervention-based criteria show higher SMO detection in each setting compared to organ dysfunction-based criteria. The CFRs of the potentially life-threatening populations (fulfilling only disease-based criteria) in low-resource settings remain high (Tanzania 13/123, 10.6%; Malawi 35/336, 10.4% versus 23/2308, 1.0% in the Netherlands). This implies that there is hardly any ‘over-inclusion’ in such settings and that these women should be picked up as SMO in the ‘potentially life-threatening phase’ of their conditions.

The lack of laboratory and clinical diagnostics for detecting organ dysfunction explains underreporting in low-resource settings [6‐9]. Similar detection rates for Tanzania and the Netherlands may seem contradictory because advanced technology in the highly resourced Dutch setting would be expected to lead to a higher detection of SMO. An explanation could be found in the supplemented clinical criteria (such as acute cyanosis, gasping, loss of consciousness etc.) as part of the local Tanzanian inclusion criteria (Table 1). These compensate the lack of extensive intensive care monitoring needed for detection by organ dysfunction-based criteria. This would also explain the low detection numbers in Malawi due to the mainly disease- and intervention-based local inclusion criteria.

Different criteria for SMO used in the three cohorts are the most important limitation of this study. SMO cases, as identified differently by local criteria, are being compared according to a single WHO MNM tool. The consequence may be an underestimation of SMO in low-resource settings as Tanzania and Malawi due to limited available diagnostics. However, this limitation also stresses the fact that application of the WHO MNM tool may differ in different contexts.

Another major issue is that, although WHO uses a threshold of five units, there is no consensus about the number of units of blood transfused, which identifies organ dysfunction [6‐9]. After including every woman in a low-resource setting who received even one unit of blood, results show a more equally distributed ‘life-threatening group’ in all settings, emphasizing that the shortage of blood for transfusion remains a large problem in many low-resource settings [13]. Also, SMO detection rate increased from 38.2% to 46.0% of all SMO cases. This 7.8% increase consists of 228 Tanzanian women (91.9%) and 206 Malawian women (53.4%). This leads to a more realistic comparison between high- and low-resource settings, because PPH is an important cause of SMO and lack of blood compounds this problem [11, 14]. Unfortunately, this is also due to unwillingness and impossibility of relatives to donate, and inadequacy or lack of blood bank storage facilities and transport [6, 7, 11, 15].

Although it is clear that there is an urgent need for monitoring health care delivery in both high- and low-resource settings, it remains difficult to determine which set of criteria should be used. In our opinion, disease-based criteria remain important in all settings, since detection rate is high and does not depend on local protocols. In contrast, for the same reason, intervention-based criteria (such as ICU admission) are of limited use. To prevent ‘over-inclusion’ for disease-based criteria, especially in high-income countries, more strict operational definitions (such as the blood loss threshold defining ‘severe postpartum haemorrhage’) are needed. For low-resource settings, supplemented clinical markers such as gasping, oliguria or jaundice could be included. Also, the threshold of received units of blood should be lowered for organ dysfunction-based criteria [8].

In conclusion, we have shown that applying solely organ dysfunction-based criteria may lead to underreporting of SMO. Therefore, a tool based on defining MNM only upon establishing organ failure is of limited use for comparing settings with varying resources. It is important to enact the discussion and eventually reach consensus for a tool that is usable in all resource settings and detects the highest percentage of the actual rate of SMO. We recommend refined disease-based criteria, accompanied by a limited set of (intervention- and organ dysfunction-based) criteria to set a measure of severity. We believe that with these adjustments, the MNM tool may be more valuable and could ultimately lead to more comparable assessments of the quality of obstetric health care across different settings.