The effect of an mLearning application on nurses’ and midwives’ knowledge and skills for the management of postpartum hemorrhage and neonatal resuscitation: pre–post intervention study

The study took place in two district hospitals in Rwanda: Masaka hospital in Kigali, an urban province; and Nyamata hospital located in the eastern rural province [29]. The two hospitals were selected out of 12 district hospitals in the two provinces because both had a high number of deliveries per year [30]. The two hospitals have been offering BEmONC services for more than 5 years.

Nurses and midwives working in the maternity departments of the selected hospitals, who deal with birth care were invited to participate in the study. The inclusion criteria for nurses and midwives were set as follows: having a work experience of at least 6 months in obstetric care, being full-time employed in the selected hospital, and willing to participate in the study. The study participants included 33 midwives and 21 nurses.

The study adopted a pre–post test design to compare knowledge and skills of nurses and midwives in the management of PPH and NR at two measurement points: pre and post the SDA intervention. The study was carried out in four phases: establishing the baseline, the educational sessions, the intervention and the endline. The baseline and endline surveys were conducted pre and post the intervention, respectively. The 6-month intervention period was preceded by training of the participants on how to use the SDA. The study took place between July 2019 and April 2020. Figure 1 provides more details on this study design.

The SDA is an mLearning application designed to provide training for health care providers in low-income countries on how to manage obstetric and neonatal emergencies. The SDA conveys knowledge and skills, such as how to ventilate a newborn in need of resuscitation or how to remove a retained placenta by means of visual guidance using animated instructional videos and a self-explanatory learning platform. The SDA also contains a catalog with essential obstetric drugs and equipment. The content of the SDA is based on WHO clinical guidelines on BEmONC and has been validated by an international group of global health experts [31]. All features and functions in the SDA are designed for low-literacy and low-income settings and work off-line once downloaded. It can thus be adapted to local contexts and policies. The SDA can be downloaded free of charge for iPhone at https://​itunes.​apple.​com/​dk/​app/​safe-delivery/​id985603707?​mt=​8 and for Android at https://​play.​google.​com/​store/​apps/​details?​id=​dk.​maternity.​safedelivery&​hl=​en.

The primary outcomes of the study were knowledge scores and skills scores of nurses and midwives. Self-administered key feature questionnaires developed and tested by Maternity Foundation were adopted, only questions regarding PPH and NR were considered. Knowledge and skills scores were measured for 2 BEmONC services, PPH management and neonatal resuscitation. The research instrument included questions on PPH management and NR knowledge and skills, demographic information, as well as exposure to in-service education on the topics under study. The knowledge questionnaire consisted of clinical content questions on causes, consequences, prevention, treatment and management of PPH and NR. The knowledge questionnaire includes 36 knowledge questions for PPH management and 37 knowledge questions for NR with single and multiple responses options. The skills were evaluated by means of a self-administered questionnaire with simulated clinical cases due to technical difficulties of evaluating skills by change in real practice. The skills questionnaire comprised six clinical scenarios simulating PPH cases for PPH management and four clinical scenarios simulating NR cases for NR with single and multiples responses options. The collected demographic data of the participants included age, gender, education level, years of experience in obstetric care, hours spent per week in providing obstetric care, and number of deliveries participated in monthly. In addition, we asked participants if they have received any in-service training about PPH management and/or neonatal resuscitation during the 6 months prior or during the SDA intervention period. This question helped the researcher to understand more about the influence of the SDA or other available in-service training. We also asked participants if they have previously used smartphones to capture their experience with smartphones.

A total of 54 nurses and midwives completed either the English or the French form of self-administered questionnaires at baseline and at 6 months post-intervention. English and French languages are both professionally used in Rwanda. Baseline measures were taken prior to the training of nurses and midwives on the use of the SDA and included: demographic data; knowledge of nurses and midwives in the management of PPH and NR; and skills of nurses and midwives in managing PPH and NR. All data were collected on paper in a classroom setting. Six months after the SDA introduction, knowledge and skills were measured a second time in the classroom setting using the same data collection instruments.

The data were entered and checked for errors in Microsoft Excel and subsequently transferred and analyzed in Stata version 16 (StataCorp LLC). Descriptive summary statistics were computed on demographic data including age, gender, education level, years of experience, hours spent per week in providing obstetric care, delivery participated in monthly, previous use of smartphone, and in-service training for 6 months prior and during SDA intervention. Since the study targeted all nurses and midwives working in the maternity department of the selected hospitals, all the eligible participants in the maternity department, a sample of more than 30 participants was required to support the use of parametric statistical tests.

This study has been approved by the Human Research Ethics Committee of the University of the Witwatersrand (M190258) and the University of Rwanda, College of Medicine and Health Sciences’ Institutional Review Board (No. 377/CMHS IRB/2018). Written consent was obtained from each participant. Confidentiality of the participants was maintained at all times. Questionnaires were number-coded thereby keeping the identity of the participants anonymous.

A total of 54 participants, 33 midwives and 21 nurses completed the surveys. More than a half were from Masaka district hospital (n = 30, 56%), the majority were female (n = 33, 61%), their average age was 33 years (SD = 7.1). The level of education was predominantly the advanced diploma (A1) in midwifery with 27 of 54 (50%) participants having A1 level in midwifery. The least represented level of education was the secondary school level (A2) in nursing with only one participant with A2 level in nursing. The majority of the participants had less than 6 years of experience in obstetric care (n = 32, 59%), spent more than 10 h per week providing obstetric care (n = 40, 74%) and participated in more than 10 deliveries per month (n = 41, 76%). Three participants have received BEmONC in-service training during the 6 months of the SDA intervention. Four participants had never used a smartphone before the study. More details are shown in Table 1 below.

The mean difference in PPH knowledge from pre- to post-test was statistically significant; mean difference = 17.1% (SD = 8.8, 95% CI 14.7–19.5); p < 0.001. The findings suggest a large effect size for PPH knowledge scores differences, Cohen’s d score = 1.8. Similarly, the mean difference in NR knowledge from pre- to post-test was statistically significant; mean difference = 19.1% (SD = 9.9; 95% CI 16.3—21.8); p < 0.001. The findings suggest a large effect size for NR knowledge scores differences, Cohen’s d score = 1.7. We found a significant association between the SDA intervention and nurses’ and midwives’ knowledge on both PPH and NR knowledge, 6 months after baseline (Table 2).

The mean difference in PPH skills from pre- to post-test was statistically significant; mean difference = 2.6% (SD = 5.9, 95% CI 1.1, 4.3); p = 0.002. The findings suggest a small effect size for PPH skills scores differences, Cohen’s d score = 0.3. Similarly, the mean difference in NR skills from pre- to post-test was statistically significant; mean difference = 5.5% (SD = 6.9; 95% CI 3.7, 7.4); p < 0.001. The findings suggest a medium effect size for NR skills scores differences, Cohen’s d score = 0.7. We found a significant association between the SDA intervention and nurses’ and midwives’ skills on both PPH and NR skills, 6 months after baseline (Table 3).

The comparisons of PPH and NR mean knowledge and skills scores by in-services training (6 months prior the intervention and 6 months during the intervention period) across the baseline and endline groups showed that there were no significant differences along in-service training lines in the pre- and post-test for both PPH and NR (Table 4). Analysis of knowledge and skills scores by previous smartphone experience did not show a significant difference for PPH and NR scores on both pre- and post-tests (Table 4). Although participants who had smartphone experience scored slightly higher on the PPH skills post-test, there was no significant difference. Similarly, there was no significant difference in mean change in PPH and NR knowledge from pre- to post-test among those experienced with smartphones and those who had never used them. Moreover, participants were classified into three groups according to their years of experience in obstetric care: group 1 (1 year to 5 years, n = 32); group 2 (6 years to 10 years, n = 15), group 3 (above 10 years, n = 7). The ANOVAs indicated significant group mean differences only for the NR skills pre-test (F (2,51) = 5.31, p = 0.008) and for the NR skills post-test (F (2,51) = 4.83, p = 0.012) (Table 5). As shown in Table 6, a Tukey post hoc test revealed that pre-test NR skills scores were statistically significantly higher in the group 1 compared to the group 3 (p = 0.006). Similarly, post-test NR skills scores were statistically significantly higher in the group 1 compared to the group 3 (p = 0.011). Also, post-test NR skills scores were statistically significantly higher in the group 2 compared to the group 3 (p = 0.023). However, there were no statistically significant differences between the pre- and post-test NR skills scores in the group 2 compared to the group 1 (p = 0.641 and 0.999, respectively). Also, there were no statistically significant differences between the pre-test NR skills scores in the group 3 compared to the group 2 (p = 0.058).

The study has three limitations. First, it is limited by the small sample of 54 nurses and midwives, which reflects the fact that the SDA intervention was implemented only in two district hospitals due to economic barriers. Therefore, the current study will not generalize findings. Second, we acknowledge the limitation of findings when assessing skills using a self-administered questionnaire with simulated clinical cases. This was due to technical difficulties of evaluating skills by change in real practice. Third, we acknowledge that some of the improvements in knowledge and skills observed in this study may arise from exposure to other learning materials other than the SDA during the intervention period. Also, the increase in knowledge and skills does not necessarily equal better outcomes for mothers and newborns which is the ultimate goal. The findings in the current paper are part of a large study evaluating the feasibility of the SDA in Rwanda. Findings of the effect of the SDA on maternal and newborn outcomes will be discussed in another paper.

Nevertheless, the evidence that the SDA intervention is effective in improving knowledge and skills of nurses and midwives in the management of PPH and NR sets the stage for expanding coverage and further testing of the SDA intervention to other hospitals in Rwanda. Additionally, the accuracy of our findings is increased by supporting the knowledge and skills surveys with focus group discussions with nurses and midwives, to provide context to and clarify reasons behind some participant responses. The findings of the focus group discussions will be presented in a separate paper.