The obstetric care subsidy policy in Burkina Faso: what are the effects after five years of implementation? Findings of a complex evaluation

The study undertook a complex evaluation using a realist evaluation approach [7]. The complexity of the evaluation relates to the dynamic and non-linear nature of the policy intervention being studied. The group began by formulating a conceptual framework, which helped to define the key relationships for which data should be sought [7]. The research tools were designed around trying to measure changes in key nodes, as far as was possible given that baseline data for key variables was often not available and policies were introduced nationally (so without control groups). The nodes were fields within the theory of change which were predicted to be important in influencing the effectiveness of the policy (for example, changes to staff motivation, or to the costs of targeted services for women). Fixed mixed methods were used to gather data, using a convergent parallel design, with research conducted concurrently and triangulated in the synthesis phase [8]. Data were then integrated to answer the key evaluation questions [7]. Theevaluation in Burkina Faso was carried out in six purposively selected districts, as well as at national level, using a case study approach [8]. Each district presents a case study which can be compared with others, not only within the country but also across the four study countries.

For further details on the overall study, please see the FEMHealth protocol and overall findings, as well as the study protocol for Burkina Faso (which contains the research tools) and the full country report of findings [6, 9‐11].

As one of the objectives was to determine the effect of the policy on health service use including caesareans, eligible districts were chosen from amongst those which performed at least 50 caesarean sections per year before the start of the policy. Fewer than 50 caesareans per year can be a sign of severe dysfunction of the surgical theatre (no surgeon, no equipment) and could be a confounding factor when evaluating the impact of the abolition of user fees.

In order to include a range of representative districts, with respect to health facilities’ accessibility (geographical and economic) and service supply and use, the districts were first classified based on four characteristics: caesarean rate, proportion of assisted deliveries, average distances to a health facility, and poverty rate [12]. Out of the 63 health districtsin Burkina Faso, data before the policy implementation were available in 52 districts (because 11 districts were newly created). Out of the 52 districts, 24 performed more than 50 caesarians per year before the policy implementation. The classification yielded four groups of which one was composed of seven districts all performing less than 50 caesarians per year before the policy. Thus, the study districts were selected from the three other groups. Two districts were selected within each group, giving a total of six districts. Four districts (Bogandé, Houndé, Orodara, Yako) with a district hospital (1st level of care) were selected out of 15 eligible and two districts (Banfora, Gaoua) with a regional hospital (2nd level of care) were selected out of nine eligible (Fig. 1). The study sites included five of the 13 health regions in the country. See Table 1 for a summary of the groupings.

Data collection was conducted from May to December 2012, using different tools, including structured questionnaires (1609 household interviews and 130 health worker surveys), medical records extraction (1752), key informant interviews (57, mainly in district and regional hospitals with facility in-charges, administrators, health workers in maternity and surgical wards, beneficiaries and in peripheral health centres with leaders and community representatives), and analysis of secondary data (from 1988 to 2010). Prior to data collection, all tools were pretested in Boromo District, which was not a study site, in October 2011 and January 2012. Table 2 gives an overview of the research methods used.

All completed questionnaires and templates were checked by supervisors and thereafter entered (using double entry) into an EPIDATA entrymask by two experienced data clerks. The database from the two data clerks was cross-checked for inconsistencies, in which case questionnaires were checked to ascertain the right information. The data analysis consisted of determining the mean, quartiles, minimum and maximum for cost data and determining the proportion for categorical data.

The data from the household interviews were analyzed according to the district, health care level, type of delivery (Caesarean, complicated, uncomplicated), and wealth quintiles. The wealth quintiles were created using the method reported by [13]. We used the most recent data available from the Demographic and Health Survey –DHS (http://​www.​measuredhs.​com/​). Common variables between DHS data and data from the structured interviews were identified, and a series of dichotomous variables (yes/no) were created for both the DHS and the interviews. A regression was performed using the DHS regression score from women’s data as the dependent variable and the variables of the identified goods as the independent variables. The correlation between the DHS score and the regression score was estimated as 0.978 (<0.001). The fitted values of the regression formed the basis of the weighted wealth quintiles and the cut points of the quintiles identified for use in the exit interview data. The analyses were performed using SPSS Version 20 software.

Data from four consecutive DHS (1993, 1998–99, 2003 and 2010) were combined to provide trends in utilisation of facility-based delivery care and caesarean sections over the period 1988–2010. The DHS collected information relating to all live births in the 5 years preceding the survey. In cases where a delivery resulted in a multiple birth, the place and mode of delivery was taken to be that of the latest-born infant. Segmented linear regression was used to calculate annual rates along with 95 % confidence intervals stratified by urban-rural or relative wealth, taking into account the complex survey design [14].

For the qualitative data the interviews with the key informant persons were recorded, transcribed and coded with N-Vivo software for thematic analysis [15]. A coding tree was elaborated prior to the analysis based on the research questions (deduction) and was reviewed and adapted with new codes during the analysis and the reading of the transcriptions (induction).

To assess the quality of care, we developed the concept of health care near-miss which measures the occurrence of omissions, delays and treatment failure [16, 17]. Health care near-misses were defined as negative events or omissions which occurred in the process of care but which did not necessarily lead to serious harm. One of our hypotheses was that a large increase of women reaching facilities could compromise safety and lead to medical errors. We calculated a score of omission (for vaginal delivery, Caesarean and care of the newborn) using between five and seven key clinical acts for each type of care. Thus, the omission score (Table 3) was calculated as the average of the proportion of women/new born for whom each key clinical act was not performed [16, 17]. The score of omission varied from zero (best quality) to seven (worst quality).

We also calculated a score of policy implementation (for uncomplicated, complicated, and caesarean deliveries) in order to assess if the quality of care was associated with the degree of policy implementation success (Table 3). This score was obtained by ranking hospital facilities based on the mean difference of the costs borne by households minus the theoretical amount they should have paid under the policy (taking into account other local initiatives for reducing fees). Items considered for estimating the total delivery cost to household included consultation, drugs and consumables, surgery (if any), transportation, payment for the delivery itself, delivery kit, surgery kit, post-operative kit, laboratory tests and ultrasonography, as well as any expenses borne outside the health facility, such as the purchase of drugs which were out-of-stock in the health facility pharmaceutical store. Theoretical amounts borne by households should have been USD22.26, USD7.29, and USD1.82 for a caesarean, a complicated delivery and a normal delivery, respectively (USD1 = CFA494.138, 2014 yearly conversion rate, http://​www.​oanda.​com/​currency/​historical-rates/​). All policy implementation scores for each type of delivery were then aggregated and a mean score for policy implementation was estimated and facilities ranked accordingly from one to six (Table 3). A lower score of policy implementation means that the mean costs borne by the household for delivery care were nearer to the official rate prescribed by the national subsidy policy.

Finally, we calculated a score for the availability of EmOC services and key service inputs (Table 4), which would help to better interpret the results for the quality of care [18]. A theoretical score was first calculated assuming a continual availability of services, personal and equipment. Thereafter, the score was adjusted taking into account the number of days where the operating theatres were not functional and the number of days of drugs stock-outs. A low score of availability of EmOC services and inputs is an indicator of lack of functionality (operating theatres not functional, absence of key personnel, absence of drugs and anesthetics etc). A low score could explain the poor implementation of the national policy, but a contrario poor implementation of the policy with a high score of EmOC services availability would highlight problems of non-compliance with the policy by the staff (frontlines workers and/or health district team).

Ethical approvals were obtained from the national ethics committee for health research of Burkina Faso, the London School of Hygiene and Tropical Medicine (UK), and the Institute of Tropical Medicine Antwerp (Belgium). All participants gave written informed consent prior to interviews (for illiterate people, mainly for household interviews, consent was explained verbally in local languages before the form was signed).

The start of the policy to subsidize facility-based deliveries and EmOC was fixed by an official note for 1 October 2006 for EmOC in referral hospitals, and 1 January 2007 for normal deliveries in health centres. However, districts began on different dates, as shown by our key informant interviews: the referral hospitals of Banfora and Gaoua and district hospitals of Bogandé and Yakoin October-November 2006, the district hospital of Orodarain January 2007, the district hospital of Houndéin March/April 2007 and the health centres in January to April 2007.

According to our financial flows analysis, a government budget line provides funding for the subsidy, under the coordination of the Directorate for Family Health, which managed its implementation throughout. Resources were allocated initially according to a fixed amount per delivery type, using the annual estimate of the number of expected births, and later on the basis of the reimbursement of eligible expenditures actually incurred by health facilities. Over the first 5 years of the policy’s implementation, the budget was USD29.37 million, and disbursements to facilities over the same period totalled USD26 million. This equates to an average annual expenditure on emergency obstetric and neonatal care of about 2 % of total public health expenditure.

Analysis of survey data indicates that deliveries in health facilities have increased since the policy started, especially in rural areas and amongst women from poor households (Figs. 2 & 3). During the period 1988 to 2010, there was an underlying secular trend of a 1 % increase in the facility-based delivery rate per year (p = 0.0157); after the policy was implemented, between 2007 and 2010, there was an additional 4 % increase in facility-based deliveries per year (p < 0.0001). The increase in facility-based delivery rates was greater in rural areas than urban areas (p < 0.0001) albeit from a lower base; and among poorer quintiles compared to wealthier groups (p < 0.0001).

Between 1988 and 2010, there was no statistically significant trend in the caesarean rate per year (p = 0.7446) (Fig. 4); after the policy was implemented, between 2007 and 2010, the caesarean rate rose consistently above 1 % for the first time, however this increase was not statistically significant (p = 0.5588). There was no significant difference in trend according to wealth (p = 0.8882).

With respect to trends in non-targeted services such as in medical ward, surgical ward, paediatrics, and outpatients services, the subsidy had very little significant effect, though some cases of adverse effects were observed in our analysis of district-level routine health data, such as delays in routine services because of the demands of the targeted services and pressures from an overall lack of human resources.

The two hospitals with the highest omission scores for vaginal delivery (Gaoua and Bogande) (above 3 omissions) also had high frequency of readmission in the delivery room for retained placenta (above 10 %); and case fatality rates among women with severe obstetric complications were highest in the two regional hospitals and Bogande and Yako district hospitals, where omission scores for caesareans were also high (Table 5). Omission scores for neonatal care are similar across facilities (around 2–3), except for Hounde district hospital (around 1), which also performs very well for vaginal and caesarean deliveries omissions scores.

Overall, 40 % of maternal near-misses were from Banfora district. Most (75 %) of the maternal near-misses were referred from another health facility. In addition, 78.6 % of maternal deaths were observed in the two regional hospitals of Banfora and Gaoua.

The amounts reported by households in exit interviews have decreased significantly, in comparison with amounts found in studies conducted shortly before the introduction of the policy to subsidize facility-based deliveries and EmOC (Table 6). The mean cost before 2006 for hospital delivery from a cohort of 1014 women sampled from three district hospitals, two regional hospitals, and two university hospitals was USD55.14. That is more than twice the mean cost observed in the present study -approximately USD26.53 [19]. However, the amounts paid for health services, primarily in hospitals (average cost between USD13.69 and USD29.35), are much higher than the rates set by the policy. Some costs which should be covered by the policy have often been borne by the households (transport from peripheral health centre to hospital, drugs, laboratory tests, and informal payments).

In Bogandé, Orodara and Yako, other financing sources provide additional funds to cover partially (Orodara and Yako) or totally (Bogandé) the 20 % of caesarean cost to be borne by the users. Thus women with caesareans in these three districts should have paid 0, USD10.12, and USD8.9 in Bogandé, Orodara and Yako respectively. The mean cost supported by the households in Orodara was lower than in all other districts. However, in Bogande and Yako, where similar cost sharing initiatives operate, no reduction in household costs was found.

Households continue to find it difficult to meet the costs associated with childbirth: 7 % reported that they were unable to pay and in rare cases (1.4 % overall, with the higher rate [4.6 %] observed in Gaoua), women have had to discharge themselves from hospital early on economic grounds. Only 2.2 % of women had health insurance; the great majority of them were from Orodara (6.6 %) and Houndé districts (3 %). There were no reports of households taking advantage of the policy’s provision for the indigent. Two thirds (63 %) of interviewed women were living in rural areas (a distribution quite comparable to that reported for the DHS 2010 sample), but 57.2 % of the sample included women living in households classified in the two richer quintiles (4 and 5), compared to only 24.2 % from the poorer households (1 and 2). Apparently, women from less poor households are those who are most able to reach the higher levels of the health system, in search of more skilled care, indicating that financial and other barriers persist.

The mean proportion of women or their relatives who were aware of the policy was low: 52 % on average and varying from 84 % in Yako district to 20 % in Houndé district. Respondents reported their overall satisfaction with the services they received (waiting times, quality of treatment received, costs of care, availability of drugs etc.). However, in Banfora there was significant dissatisfaction with the cost (5 %) and the availability of drugs (23 %). In addition, dissatisfaction has also been reported in relation to the cleanliness of premises (9 %), mainly at Orodara (20 %) and Gaoua (13 %).

With respect to the motivation of health personnel, the workers frequently reported an increased workload resulting from policy implementation (not just clinical but also administrative in order to comply with policy reporting requirements), and that they did not gain any direct financial benefit in compensation. However, the majority of workers exhibited good motivation, in part because of the improvement of working conditions made possible through the implementation of the policy. For example, seven out of ten health staff reported higher work motivation, partly because the policy enabled them to provide more timely and adequate care. Indirectly, the increased facility-based deliveries may have raised health personnel revenue: 41 % reported an increase in their revenue sharing over the period. Analysis of routine data on staff to workload ratios by district suggests varying patterns of change and that overall the workload remains acceptable and manageable (Fig. 5). It did not exceed four deliveries per week on average for midwives and five caesareans per week on average for physicians.

Financial analysis and interviews with staff and managers suggest that the policy has resulted in greater flexibility for facilities, by making available more financial resources and medicines. Resources were made available throughout six-monthly reimbursements of the health facilities, based on the estimated amount they incurred in treating women. However, interviews also revealed constraints that hamper the implementation of the policy, such as gaps in continuity and quality of the services (e.g. a lack of ambulances for referrals).

“…The main concern is about the transportation for the implementation of SONU [the policy] because for a lady if we declare that there are complications, we must bring her to the CMA [the district hospital]. If there is not a functional ambulance, it poses a problem” (administrative respondent)

Interviews suggest that cases of unsuccessful implementation were related to institutional constraints and non-compliant practices by some health workers, especially in Orodora district. These problems resulted from insufficient qualified staff, inadequate equipment or stock-outs of inputs (e.g. non-availability of tests or medicines in the facilities’ pharmacy), obliging users to pay for external services. Non-compliant practices refers mainly to unscrupulous service providers in hospitals, who take advantage of the ignorance of users about the policy to claim full payment for procedures, or to stockpile medicines through fake prescriptions in order to illegally resell them subsequently to unaware users.

“…these kits were a source of enrichment for a certain category of persons” (health staff respondent)

As well as analysing effects, the FEMHealth team used in-depth interviews and analysis of secondary data to understand why implementation of the policy varied across districts, using a realist evaluation approach [7]. It seems that one factor was the lack of clarity in the official guidelines of the policy, which negatively impacted on its implementation (e.g. unclear inclusion of associated pathologies or lack of precise definition of certain procedures). The adoption of the policy by the local health systems was also done with both positive and negative changes. Positive adaptation included the arrangements made in all districts to better control the use of medicines by care providers, following various non-compliant practices. There was also the pre-financing, especially in health centres, of the costs of medicines to maintain service delivery, thus overcoming delays due to reimbursement procedures. Negative adaptation included non-compliant practices (e.g. the “dubious” management of kits, prescription without real pathologies and the sale of leftover medicines to users, all for personal gains).