Evaluation of a maternal health care project in South West Shoa Zone, Ethiopia: before-and-after comparison

Maternal mortality ratio (per 100,000 live births) is estimated to have significantly declined in Ethiopia, from 1,250 in 1990 to 353 in 2015 (a 72 % drop); just shy of achieving the Millennium Development Goal (MGD) 5 target of 75 % reduction [1]. Ethiopia has also made remarkable achievements in reducing child mortality; the country achieved its MDG 4 target of reducing child deaths by two thirds between 1990 and 2015 [2]. Despite these achievements, the number of maternal deaths in Ethiopia is still high; the country is one of the ten countries that contribute to 59 % of global maternal deaths [1]. Ethiopia also has a disproportionately high number of neonatal deaths; 43 % of the under-5 deaths are neonatal deaths [3].

The high maternal and neonatal mortality reflect poor coverage of maternal and neonatal health care services, poor quality of care provided in health facilities, and inequity in access to health services. Coverage of the recommended minimum four antenatal care (ANC) visits increased from 19 % in the 2011 Demographic and Health Survey (DHS) to 32 % in the 2014 DHS survey, and that of skilled birth attendant (SBA) at delivery correspondingly increased from 10 to 16 % [4, 5]. Ethiopia is one of the six countries where more than half of the mothers and children in the poorest 20 % of the population receive two or fewer of eight essential interventions for preventing maternal and child deaths [3]. The reasons behind the high maternal and neonatal mortality in Ethiopia have been explained using the “three delays” model [6].

Various health system constraints affect maternal health service delivery in Ethiopia. These include inadequate basic health infrastructure, shortage of skilled staff, weak referral systems, limited availability of equipment, limited financing for services, weak management, poor staff motivation, and weaknesses in implementation of government programs [7, 8]. The density of doctors, nurses and midwives per 10,000 population in the country was 6.3 in 2012/2013 [9]; way below the 23 recommended by WHO [10], and the per capita total expenditure on health was 44 US$ in 2012 [3]. A substantial amount of healthcare funding comes from donors; in 2011, Ethiopia received the second highest share (6.1 %) of the total official development assistance for maternal, neonatal and child health [3].

Doctors with Africa CUAMM (http://​www.​mediciconlafrica​.​org/​), hereafter referred to as CUAMM, is an Italian non-governmental organisation (NGO) that has been supporting health service management and delivery in Ethiopia since 1984. CUAMM’s current strategy is based on the continuum of care approach [11]. Between 2012 and 2015, CUAMM implemented a multifaceted maternal and child health project in three districts (so called woredas) in South West Shoa Zone, Oromia region. The project aimed to improve access to maternal and child health services through tackling demand and supply side barriers to service access; focusing mainly on health centres (HCs) and the community. Key determinants of maternal health service access and utilisation in the districts include distance to health facilities, attitude towards maternal health care, knowledge of maternal health, perceived quality of maternal health services, involvement of the family members in decision making on delivery place, and birth preparedness [12]. Additionally, there is stack inequity in utilisation of maternal health services in the districts by wealth status and urban/rural residence [12, 13].

This study aimed to evaluate the effect of this project on access to essential maternal and neonatal healthcare services including ANC, delivery by a skilled provider and postnatal care (PNC).

This study evaluated a multifaceted maternal and child health project implemented by a non-governmental organisation. The results suggest that the project was associated with increased coverage of receipt of all three basic components of ANC and SBA at delivery, but not with four ANC visits and PNC. The effect on SBA at delivery and receipt of three basic components of ANC was probably partly mediated through better perception towards the quality of maternity care provided in health facilities, improved attitude towards maternal health and improved birth preparedness (for SBA at delivery). Coverage of SBA at delivery was driven by increased utilisation of HCs which before the intervention were largely underutilised; attending to only 7 % of total deliveries despite their large number. Receipt of three basic components of ANC can be taken as a proxy measurement of the quality of ANC, thus the improvement in this indicator suggests that the project improved access to quality ANC.

The project was biased towards strengthening HCs to provide maternal and child health care services. This is because the hospital was already being well utilised. A recent study has shown that although there is inequity in access to maternal health services in the intervention area, women utilising HCs were more likely to be poorer (and rural residents) than those utilising the zonal hospital [21]. For most women, HCs were the nearest facilities for accessing childbirth services, however, before the start of the project, these facilities were being grossly underutilised partly because of being perceived to be providing poor quality of care [12]. Over the course of the project, the percent of women who perceived that the quality of care at the nearest health facility was good more than doubled to 35 %. Thus, the results of this evaluation suggest that strengthening HCs to provide delivery services could be one of the effective ways of scaling up coverage of SBA at delivery; in line with the principles of primary health care [22].

The WHO recommends that pregnant women should attend a minimum of four ANC visits to allow for appropriate delivery of a complete package of ANC services [23]. Although the coverage of four ANC visits increased from 44.8 % before the project to 59.2 % later in the project, this increase was not significant after accounting for socio-demographic factors. This observation was unexpected given that it is easier to improve the coverage of services such as ANC and PNC that can be offered through outreaches than those, such as childbirth care, that are offered only in fixed health facilities [24]. Given that a steeper increase in ANC coverage seems to have occurred in the later months of the project, it could be that in the initial stages, more time and effort was spent on increasing childbirth in HCs through improving infrastructure, providing equipment and training health staff than on providing preventive services at HPs and in the community.

There is a big challenge in providing PNC in Ethiopia. In the 2011 DHS, only 50 % of women who delivered in health facilities received PNC within the first two days [5]. It is unclear why the coverage of PNC attendance did not change in the course of the project. In the study districts, women with uncomplicated deliveries are usually discharged after six hours, and they may not return within a few days to the health facility due to barriers, such as distance, that affect access to health services. Different strategies for PNC with varying challenges to mothers and providers exist [25]. To improve PNC coverage, in 2012, the Ethiopian Government adopted a mixture of facility-based and community-based PNC strategy, leveraging the efforts of HEWs. HEWs are required to visit the mother and the baby within 48 h after birth [26]. Despite this, a survey conducted in 2014 showed that only 0.8 % of women in Oromia received PNC within two days by a HEW [4]. In our study, only 1.4 % of the 999 women surveyed received PNC within seven days by a HEW. Thus, there is still a huge potential to increase PNC coverage through HEWs. Lack of improvement in PNC coverage could also be because the project may have placed more emphasis on intrapartum care, including provision of emergency obstetrics care at HCs, than on PNC. A community based intervention project that involved family meetings and labour and birth notification to a HEW led to 3- to 10-fold increases in PNC coverage over a 2-year period in rural Ethiopia [27]. Such an approach could be piloted in the study districts.

Some researchers have used coverage of SBA at delivery to classify health systems in Africa as follows: low health system context where SBA at delivery is less than 30 %; middle health system context where SBA at delivery is 30–60 %; and high health system context where SBA at delivery is >60 % [24]. Based on this classification, it can be argued that the project strengthened the health system to provide maternal and neonatal health services, but a lot still needs to be done to ensure continued progress and sustainability. Sustainability of donor funded projects is always of concern to the government, donors, implementing agencies and project beneficiaries. In this study, the sustainability of removal of user fees at hospital, which is private-not-for-profit, may be of concern. Nonetheless, even before the present project, user fees at the hospital were highly subsidized because the hospital was getting financial support from the government under a public-private partnership (PPP) framework [13]. CUAMM and the hospital will have to continue to negotiate with the Ministry of Health to ensure continuity of the PPP and its expansion to cover all maternal neonatal and child health services.

The project started off with higher baseline coverage indicators than the national averages probably because CUAMM had been supporting health service delivery in the districts for a while. For instance, the NGO had been supporting the running of the zonal referral hospital since the year 2001 [28]. In addition, for PNC, the higher coverage could be because we defined this indicator broadly (i.e. care within seven days of birth). The high coverage of SBA at delivery but low coverage of PNC implies gaps and/or missed opportunities in the continuum of care [29]. The findings highlight the need to investigate further why women deliver assisted by SBA but don’t receive PNC. The aim should be to mitigate missed opportunities and improve access to health care because most neonatal deaths occur during the postnatal period [30], and failure to provide high quality PNC to neonates can hamper efforts to reduce neonatal mortality.

This study has some limitations. The use of a historical control group did not allow us to adjust for secular trends in service use. Although we adjusted for socio-demographic factors that might affect health service utilisation independent of any intervention, Ethiopia has been experiencing a general nationwide increase in the coverage of maternal health services. For instance, at national level SBA at delivery increased from 10 % in 2011 to 16 % in 2014 [3, 4]. In our study, coverage of SBA at delivery more than doubled in a shorter time despite starting off at a higher baseline, which suggests that the project played a role in this. This study focused on coverage, yet coverage alone cannot reduce maternal and neonatal deaths if the quality of care provided is poor and women in the poorer strata are excluded. A study conducted at the hospital found that the quality of maternal health care at this facility was good [21], but little is known about the quality of care at HCs especially with regard to intrapartum and postpartum care. We defined PNC as care within seven days after childbirth; this is different from the international definition of PNC as care within two days. Data on PNC within two days were inadvertently not collected in the second survey. Thus, this indicator should be interpreted cautiously especially when making comparisons. Future evaluation should focus on a detailed evaluation of the quality of care at HCs given their increasing importance in service delivery as it emerged in this study. Because some of the questions asked during the interviews required women to recall events that occurred up to the past two years, our estimates may have been affected by recall bias. Finally, because this was a multifaceted project, we cannot associate any outcome with any particular component of the intervention.

The maternal and child health project implemented by the NGO Doctors with Africa CUAMM in Wolisso, Goro and Wonchi districts in Ethiopia was associated with increased coverage of receipt of three basic components of ANC and delivery assisted by a SBA. There was however no increase in the coverage of at least four ANC visits or PNC after accounting for potentially confounding factors. Increase in coverage of SBA at delivery was driven by increased utilisation of childbirth services at HCs. More efforts, involving HEWs, are needed to increase the coverage of ANC and PNC.