Evaluation of a quality improvement intervention for obstetric and neonatal care in selected public health facilities across six states of India

In collaboration with the Ministry of Health and Family Welfare, we supported 27 high priority districts in six states - Delhi, Haryana, Himachal Pradesh, Jharkhand, Punjab, and Uttarakhand. Hence, the selection of intervention sites are purposive in nature. MoH had identified these districts based on selected key performance indicators in RMNCHA domain. These comprised the bottom quintile of all districts in each state in terms of performance and also took into account equity concerns. Within these districts, we planned to have an impact on maximum number of deliveries that was possible for one coach in every district to handle. In each district, we worked at the district hospital, sub district hospital/s, and all primary and community health centres in a block selected on the following criteria: 1) willingness of block leaders to implement a QI project, 2) accessibility, and 3) mid-level performance¹ on health indicators. This site selection allowed us to work for 50–80% of child-births occurring in government health facilities in every district. In these facilities, care is provided by nurses, doctors and specialists (OBGYN and Pediatrics) who composed a QI team. Whenever required and available, additional staff such as pharmacist, helpers, ASHAs (community health workers), facility head, and auxiliary nurses were involved.

After identification of facilities, project staff provided initial training to health workers and managers on the seven steps of quality improvement, laid down in a QI Implementation guide [14]. This included an introduction to how to: 1) identify an improvement goal, 2) form and work in a team, 3) analyze current systems of care to identify the reasons for poor care, 4) develop simple measurement systems, 5) develop options for possible solutions, 6) test these interventions on a small scale, 7) implement the interventions that were found to be successful. Interventions covered systemic and process changes including facility improvement, supplies of medicines, improved record keeping and so on depending on the nature of the problem identified.

After the initial training, project staff visited facilities roughly every month to provide hands-on support to health workers in applying these skills to identify and solve problems of patient care. The project employed one coach per district to provide this ongoing support. Initially, facility staff were encouraged to choose improvement goals involving routine care (preventative, diagnostic, or management interventions) that all pregnant and post-partum women and all neonates should receive. In this paper, we look at the effect of the QI support on nine elements of routine care and perinatal mortality.

Data were self-collected by the teams of health workers and came from existing hospital records which included patient case files, ANC cards and delivery registers. These data are routinely collected as part of the ANC, delivery and post partum procedures, and we did not collect additional data. Each month, project staff verified and recorded data in a predesigned Excel database. Data were then aggregated at district, state, and finally at aggregate level, with data verification at each step. Additional files are attached to Additional file 1 indicate proportion of services (indicators) conducted, and Additional file 2 to show aggregate number of patient records reviewed for ANC, delivery and newborn care across all months. Because facilities only reported data related to the goals that they were working on, the number of facilities reporting data for each indicator varies. Data reporting started as soon as district coaches were hired and QI teams formed, from July 2013 onwards. Most of the indicators were taken up for improvement following the QI model, from January to February 2014. Data collection continued till August 2015 when the project officially closed.

The data pertain to an intervention project and are taken from hospital records that are routinely collected and hence no ethical clearance was required. This was a basic program evaluation that was a routine part of implementation with no additional data collected than would be normally expected in such a program. Anonymity and patients’/clinicians’ rights were respected and we had permission from facilities plus MOH to implement the intervention. Furthermore, as per the national guidelines for biomedical research, “research on publicly available information, documents, records, works, performances, reviews, quality assurance studies, archival materials or thirdparty interviews, service programs for benefit of public having a bearing on public health programs, and consumer acceptance studies” are waived from voluntary informed consent process [15]. The use of data for a quality improvement intervention falls in the above mentioned category.

For tracking the progress of our intervention, we selected a set of nine process indicators spread across four domains of antenatal (ANC), intrapartum, essential newborn care (ENC), and postpartum care. These indicators were chosen based on prior experience of the implementers about which elements of care are good to teaching health workers how to use QI approaches. Facility staff chose new indicators to work on as their skills grew based on what they perceived to be priorities in their facilities, which are not presented in this analysis as the start time differs. The current analysis focuses on the nine process indicators that the QI teams worked on at the start of the USAID ASSIST support. Additionally, we included perinatal mortality indicator to see the effect of the intervention. The specific indicators were:

ANC:

Intrapartum:

Essential newborn care:

Postpartum Care:

Outcome Measure:

Performance of desired antenatal care practices increased considerably over the intervention period (Fig. 1). Segmented regression results (Table 3) indicate that just before the beginning of the observation period, on average 40% (adjusted for time and intervention) of women went for Hb test. With time, there was significant month-to- month change of 0.04 in the proportion of Hb test (P < 0.014)) and even right after the cap, the estimated proportion increased abruptly by 0.17. There was no significant change in the month-to-month trend in the mean number of tests after the cap, as the trend of approximately 85% was maintained across the months of intervention. Proportion of ANC history taking increased by 0.30 over time with the intervention having a significant impact (P < 0.001). Similarly, ANC counseling significantly increased over time as well as during the time after intervention by 0.025 (P < 0.001).

Similar results to antenatal practices regression model explains the proportion of women given oxytocin within one minute of delivery, proportion of newborns kept dry & warm, sterile cord care, early breastfeeding and vitamin K provided were significantly (p < 0.001) increased by 20–70% during post intervention period and month-to-month significant (p < 0.001) improvement (3–13%) was also observed (Table 3). As can be seen from the time series chart (Figs. 2, 3 and 4), the indicators began to increase during the preparatory phase of the intervention while after February 2014, except Vitamin K, the provision of all other practices stabilized at around 90% during the early months of intervention (breastfeeding at around 85%). Vitamin K provision, however, kept increasing significantly from month to month during the period of intervention (p < 0.01). It took 8–9 months to increase vitamin K injection from 40 to 70%, after which it was sustained at 98% until the end of the project.

Postpartum care in terms of checking of mother’s vitals also improved over the intervention. The average number of times that blood pressure and pulse was checked for a woman within 6 h after delivery increased from 0.3 to 3.5 times (Fig. 3). Segmented regression analysis shows that while post intervention had a significant impact in its increase (75%, p < 0.05), there was no significant increase in the month to month trend during the period of intervention.

While perinatal mortality decreased as a function of time (P < 0.01), the pre-post intervention showed no significant impact, nor did the trend show any month to month change during the period of intervention (Table 3).

In general results from segmented model confirms month-to-month and pre-post intervention period significant improvement in health care indicators whereas trend after intervention did not have the same effect in variation (barring some indicators) which is probably due to a rapid and huge increase in the early phase of intervention following which many of the indicators stabilized at a constant rate.

The results clearly demonstrate that teams can use QI approaches to redesign care delivery processes and improve delivery of antenatal care. Most of these improvements involved reorganizing work stations and processes, coupled with improved recording practices aimed at early identification and management or referral for high-risk conditions. For example, in facilities in Himachal Pradesh, ANC triage was improved through redesign of space and task shifting of staff. Previously, crowds of ANC patients would rush into a hall, where depending upon the patient load and number of staff, they may or may not receive the requisite care. After QI implementation, stations and staff were defined for specific activities, which led to less crowding and fewer missed opportunities. Simultaneously, recording of clinical care practices improved [27] Reporting of hemoglobin measurement and obstetric history taking was done diligently, since these indicators were considered vital for identification of high-risk pregnancy and therefore we see a rapid increase in these processes in the early months of the intervention after which they plateaued. We did not see the same for ANC counseling, exhibited by fluctuating trends, which is considered a routine and mundane job by most staff. It was only after 12 months that counseling showed a gradual improvement; this occurred due to renewed focus on counseling and introduction of guidelines. Our findings are in alignment with another study in Kenya which demonstrated improved adherence to clinical standards of antenatal care, such as measurement of hemoglobin and blood pressure, administration of tetanus toxoid vaccination, and counseling following a QI intervention [28].

We found swift improvement in processes that do not require additional technical skills, are delivered as a single step, and are one-sided (do not involve client compliance issues) such as giving oxytocin within one minute of birth, vitamin K, cord care, and keeping babies dry and warm. For example, improvement could already be seen in oxytocin administration as teams were preparing for the intervention (from 1% in July 2013 to 57% in February 2014). Most facilities were able to apply simple changes like prefilling syringes with oxytocin and disseminating information among staff conducting deliveries to improve administration, and therefore we see a quick increase during the early months of intervention up to May 2014 when it reached 90% and stabilized thereof. Our finding echoes that of another study in Rajasthan where a QI intervention helped increase delivery of oxytocin after birth from 57 to 90% [29]. Early breastfeeding increased over the intervention period (Table 3). The time series chart at the all state aggregate level shows a high reporting in baseline which actually drops once the intervention starts, probably a result of more accurate reporting, picking up again during intervention after which it does not show any increase month to month during intervention period (Fig. 2). Not reaching 90–100% can perhaps be explained by the fact that out of all deliveries, there are some which result in extremely sick newborns or sick mothers who are unable to breastfeed. Keeping babies dry and warm increased to cover almost all babies at project end, which was higher than the reported 80% coverage from the Rajasthan QI study [30]. It could be that our intervention ran for a longer period of time (21 months) during which new steps in improvement had time to become established practices. Facilities were quickly able to raise vitamin K administration to approximately 90%. Achieving the final 10% took longer, as teams had to learn how to give vitamin K to neonates with complications, which required additional care.

Although there was increase in the number of vitals check after delivery, not all states demonstrated this increase and hence we did not have statistically significant improvement. Nevertheless, the increase in certain states health staff was achieved by setting a schedule of monitoring vital parameters in the patient’s records and documenting it for all cases. In some facilities, vitals monitoring was improved by reorganization of the labor ward and partnering with family members to inform nurses in case of danger signs [30]. This shows that despite shortage of staff, which is reported as the main limitation in carrying out monitoring of heart rate and blood pressure [3], improvement can be made. Specific changes and strategies associated with improvements in all these elements of care were gathered and are currently available online [31].

Although perinatal mortality showed a decline over time, there is no evidence that the intervention had a significant impact on its decline. This may be because there was an insufficient baseline period or the intervention period was not long enough to affect mortality. Alternatively, the decline we observed could simply be due to secular trends.

There are a number of limitations in our study. The number of facilities varied across the intervention, especially those reporting ANC indicators. Circumstantial reasons such as the departure of a district coach or the implementation of a government program that moved staff to other tasks or achievement in the said improvement goal led to less reporting. Nevertheless, we believe that such factors were not extensive enough to influence the overall trend since the number of facilities was still sufficient in terms of in-patient loading to measure improvement during entire intervention period (Table 1). While data were generally obtained from hospital registers, some data were based on observations of 20 patients or a sample of records. This, however, occurred in the initial stage of the intervention. As the said indicator became an established practice, data from hospital registers became the norm. Furthermore, since we did not have sufficient resources to conduct separate interviews with patients, we do not have their perspective of whether and how clinical care was delivered which may affect our interpretation. Future programmes must include user perspective. Another limitation is that while we analyzed data at the state level, we have not presented individual state results due to limitation in text length. Lastly, we could not estimate the effect of external variables, such as the effect of other interventions to improve care. However, no drastic changes to the macro-level factors (policy and program) occurred in the 26-month period under reporting. We therefore believe that this study supports the effectiveness of QI intervention in improving patient care.