Reliability of maternal recall of delivery and immediate newborn care indicators in Sarlahi, Nepal

The intrapartum and immediate postpartum periods are a time of high mortality risk for newborns and mothers. Neonatal deaths account for almost half of all under-five deaths, and mortality within this period has been difficult to reduce [1]. Numerous interventions exist to minimize risk to both newborns and mothers in the peripartum period. Data on population coverage of these interventions are needed for health programs to ensure that interventions are reaching those in need. However, such data are often scarce or unreliable. Information on the content or quality of care administered during labor and delivery at health facilities is frequently un- or under-documented, and rarely reported in national health information systems. Even less information is available for deliveries occurring outside of the government health system. Population-based surveys of women, with questions focused on care received during recent deliveries, are often used to capture data on intervention coverage for both facility and non-facility deliveries. However, an increasing number of studies have shown that women are often unable to accurately report on the content of care received, particularly for interventions occurring during the peripartum period [2‐6]. A qualitative study by Yoder and colleagues in Bangladesh and Malawi found mothers had difficulty understanding some terminology related to peripartum care and comprehending questions about the timing of events following birth [7]. Work by McCarthy and colleagues suggests that pain, fatigue, and relief of a successful delivery may distract women from noting the care they received [5]. Further, women may not be told about the care they receive, such as the type of injection, or maybe more or less likely to report an intervention due to social desirability biases, sometimes providing responses that are believed to be viewed more positively by others. To date, validation studies of peripartum care have focused on women delivering in health facilities and have primarily been conducted in sub-Saharan Africa or Latin America.

We assessed the reliability of maternal report of delivery care and immediate newborn care for a sample of both home and health facility births in Nepal. Within the first 3 days after delivery, mothers were asked to report on interventions received during the peripartum period. These same women were visited between 1 and 24 months later and asked to recall the interventions they received during the peripartum period. Maternal report at both time points was compared to assess the reliability of maternal recall of peripartum health interventions. We also assessed potential variation in recall reliability by characteristics of the birth, the mother, and intervention prevalence.

Of the 1892 women selected for the reliability study, 363 were not available, 5 had moved permanently, 3 had died, 4 refused to participate, and 1517 were consented and interviewed (see Supplementary Figure 1). There was no significant difference in the characteristics of those women who were and were not available to participate. After excluding 15 participants (birth assessment > 72 h after birth [n = 3], twin delivery [n = 1], repeat participation due to multiple eligible births [n = 11]), 1502 mother/child pairs were included in the substudy. Of these, 220 were enrolled in the one-month recall group, 207 in the three-month group, 205 in the six-month group, 196 in the nine-month group, 193 in the 12-month group, 284 in the 18-month group, and 197 in the 24-month group (Fig. 1).

More than half of newborns were male (55.5%), and a majority of births occurred in the home (53.8%) (Table 1). The mean recall period was 10.8 months. The mean age of mothers was 23.9 years at the time of delivery. Most mothers had no schooling (68.3%) and had prior children (71.4%). Participants were nearly universally of Madhesi (people of the plains) ethnic origin (96.2%), frequently lacked a household latrine (71.2%), but owned some type of land (97.4%). The substudy sample was comparable to the parent trial sample, but the parent trial sample was more balanced by child sex (male = 51.3%). Stratifying by the site of delivery, women who delivered in health facilities were younger and more educated than women who delivered at home (see Supplementary Table 1). They also were more often having their first child (41.6% vs 17.3%) and more often reported birth complications (28.9% vs 8.7%) compared to women who delivered at home.

As an initial assessment of intervention recall and question comprehension, we assessed the proportion of mothers that responded DK when asked about each intervention or practice at the initial post-birth assessment and at follow-up (Table 2). Only a handful of indicators (4 for facility births, 1 for home births) had a greater than 5% DK response rate during the initial assessment, but this increased to 9 and 5 indicators, respectively, at follow-up. Recall of the type of injection received the highest proportion of DK responses. During the initial interview, most mothers reported receiving multiple injections during delivery, and mothers could not identify approximately 90% of the injections they reportedly received. Recall of type of injection was also poor at the follow-up survey, however it was partially masked by a reduction in the number of injections that mothers reported receiving during delivery. During the initial assessment, women reported receiving 1.83 injections on average, which fell to 1.27 during the follow-up survey.

The proportion of women who gave DK responses was higher overall among women that delivered at a facility, compared to women who delivered at home, at both the initial and follow-up interview. For example, at the initial assessment, 10.4% of women who delivered at a facility could not report whether a new blade had been used to cut the umbilical cord compared to just 0.2% of women who delivered at home. This was true for other cord care indicators, with 5–10% of facility-delivering mothers reporting DK at the initial assessment increasing to 15–29% reporting DK at the follow-up survey. Indicators involving the timing of wiping, wrapping, bathing, and cord-cutting had > 5% DK responses at follow-up across both home and facility births, but the proportion of DK responses was much higher for facility births.

The reported intervention coverage, percent agreement, and kappa values of each indicator are presented in Table 3, and for facility and home deliveries separately in Table 4. The majority of indicators had a high observed percent agreement, but most kappa values were below 0.4, indicating agreement was primarily due to chance. Among all observations, three indicators showed moderate reliability with kappa values greater than 0.4, including “any part of the clean birthing kit used for the delivery,” “sheet from clean birthing kit used for the delivery,” and “cord cut after placenta delivered.” Only one indicator, “received any injection during delivery,” demonstrated high reliability with a kappa of 0.737. Stratifying by place of delivery, the reliability of maternal report was much lower among women who delivered at a facility. Only 5 out of 14 indicators had greater than 70% agreement among facility deliveries, and none had kappa values above 0.4. Among home deliveries, 11 out of 14 indicators had > 70% agreement, and both use of a clean birthing kit and any injection during delivery had kappa values above 0.4 and 0.6 respectively. Inclusion of DK as a separate response category did not significantly alter the reliability of any indicator, with the exception of “received oxytocin / ergometrine during delivery,” where reliability improved due to the high number of DK responses (see Supplementary Tables 2 & 3).

We assessed associations at the indicator level between underlying intervention coverage or prevalence, based on the initial maternal report (after excluding DKs), and measures of maternal recall reliability. There was a U-shaped association between intervention prevalence and observed percent agreement (Fig. 2). The proportion of women who changed their initial report of each indicator and coverage of each intervention or prevalence of each practice in the population is presented in Supplementary Table 4. We observed an association between underlying intervention coverage and the proportion of women over or under-reporting the intervention at the follow-up interview (Fig. 3). Among both home and facility births, we observed a higher proportion of women changed their initial report of no intervention to received intervention (over-reporting) for interventions that were more common in the population. Conversely, we observed a higher proportion of women changed their initial report of received intervention to did NOT receive the intervention (under-reporting) among interventions that were less common in the population. These associations were stronger among home deliveries than facility deliveries. Underlying intervention prevalence accounted for 83% of the variation in over-reporting and 85% of under-reporting among home births, but only 46 and 43% of over- and under-reporting respectively among facility births. There was no association between underlying intervention prevalence and indicator kappa statistics (data not shown).

We observed no significant differences in percent agreement by binned recall time for any of the four best performing indicators (Fig. 4). Similarly, in the adjusted logistic regression model, we observed a negligible but statistically significant reduction in agreement of report by increasing recall period for most indicators, controlling for child sex, place of delivery, birth complications, parity, maternal age, education, and ethnicity (Tables 5). Mothers had statistically lower odds of reliably reporting 10 of 14 interventions if they delivered in a facility and statistically greater odds of reliably reporting if the baby was wrapped before the placenta delivery if they delivered at a facility. After disaggregating by location of delivery, there was little statistical difference in the reliability of report by recall length among home deliveries (Table 6). There were no clear cross-cutting associations between respondent and birth characteristics and over- or under-reporting overall (see Supplementary Tables 5 & 7) or by site of delivery (see Supplementary Tables 6 & 8).

This study found poor reliability of maternal report of immediate newborn care indicators as collected through a household survey. A high percent agreement between initial and follow-up reports was observed for most interventions. However, the kappa values for most interventions were low, suggesting observed agreement was mostly due to chance from very high or very low intervention coverage. This was further evidenced by the U-shaped association between intervention prevalence and observed agreement. Only receipt of an injection during delivery could be recalled with high reliability, but not information on the type of injection. For most indicators, women who delivered at home had greater odds of reliably reporting on the intervention compared to women who delivered at a facility. We also observed a high proportion of women who delivered in health facilities failing to recall interventions during the initial interview (< 72 h) after delivery. A negligible, although sometimes statistically significant, decline in recall reliability with increasing time since birth was observed for most indicators.

This study reinforces previous research that suggests mothers are unable to effectively recall interventions or practices which occur during the peripartum period. Studies by Blanc, McCarthy, and Stanton have demonstrated poor recall accuracy within the peripartum period among women in multiple low- and middle-income settings [2‐6]. Multiple factors could contribute to poor recall. Our study suggests that mothers may never have known about some interventions they received – as evidenced by the high proportion of mothers that reported they didn’t know the type of injection they received among both facility and home births at the initial assessment. This agrees with findings from Mexico and Kenya, which showed poor recall of peripartum interventions among women at initial discharge from their labor and delivery facility [2, 3]. Similar to previous studies, we found recall close to the time of care was generally poor; however, there was little evidence of reliability-altering recall degradation with increasing time since delivery up to a two-year recall period [5, 6]. Currently, the DHS asks women to report on delivery and newborn care for their most recent birth in the previous 3 years, with surveys prior to Phase 8 asking about births in the previous 5 years. However, this study and others only assessed recall for up to 2 years postpartum.

Previous studies have primarily assessed recall among women who delivered in health facilities. In general, this study found the proportion of women who were unable to report whether they received an intervention within 72 h of birth was higher among facility deliveries relative to home deliveries. A possible explanation is that women delivering in a facility may not have been informed about the details of interventions received, such as the substance applied to the child’s umbilical cord or whether the instrument used to cut the cord was new or had been sterilized. We also observed significantly weaker recall reliability for most indicators among women who delivered at a facility compared to women who delivered at home. This suggests receipt of these interventions were less salient events for women delivering in facilities potentially because they were not informed or counseled on various interventions, events may have been obscured by an unfamiliar or chaotic environment, or they paid less attention to events because they trusted the skilled attendants providing care. Alternatively, mothers may have had a better rapport with home birth attendants, often family or other community members, who may have more effectively communicated events occurring throughout the delivery process. No other characteristics had a consistent effect on mothers’ recall reliability.

We observed an association between underlying intervention coverage and maternal report. Interventions that were more common in the population were more likely to be over-reported during the follow-up interview. Likewise, interventions that were uncommon in the population were more likely to be under-reported. This association was stronger among home births than facility births. A forthcoming assessment of maternal report of antenatal and postnatal care interventions in Kenya, Cambodia, and Bangladesh found a similar association between intervention prevalence and under- and over-reporting of intervention receipt among women receiving facility-based care [8]. This association could potentially reflect a social desirability bias, such as wanting to report a practice the mother thought the interviewer would perceive favorably. Alternatively, if the mother could not clearly recall the intervention, she may assume that an intervention did or did not occur if it was or was not a standard practice in the setting.

This study reinforces previous findings that mothers are unable to effectively report on many interventions or practices within the peripartum period. Household surveys which rely on maternal report therefore may not be an appropriate method for collecting data on coverage of many interventions during the peripartum period. This is particularly true among facility births, where many interventions may occur without the mother’s full knowledge. New methods are needed for generating more robust estimates of peripartum intervention coverage. Data suggest that mothers are able to accurately report on the location on delivery. Linking valid data on where mothers deliver with robust data on the content and quality of delivery care at these facilities may be used to generate estimates of intervention coverage.