Figures
Abstract
Objective
To assess the accuracy of assessment of eligibility for early medical abortion by community health workers using a simple checklist toolkit.
Methods
Two hundred seventeen women in Ethiopia, 258 in India and 236 in South Africa were enrolled into the study. A checklist toolkit to determine eligibility for early medical abortion was validated by comparing results of clinician and community health worker assessment of eligibility using the checklist toolkit with the reference standard exam.
Results
Accuracy was over 90% and the negative likelihood ratio <0.1 at all three sites when used by clinician assessors. Positive likelihood ratios were 4.3 in Ethiopia, 5.8 in India and 6.3 in South Africa. When used by community health workers the overall accuracy of the toolkit was 92% in Ethiopia, 80% in India and 77% in South Africa negative likelihood ratios were 0.08 in Ethiopia, 0.25 in India and 0.22 in South Africa and positive likelihood ratios were 5.9 in Ethiopia and 2.0 in India and South Africa.
Conclusion
The checklist toolkit, as used by clinicians, was excellent at ruling out participants who were not eligible, and moderately effective at ruling in participants who were eligible for medical abortion. Results were promising when used by community health workers particularly in Ethiopia where they had more prior experience with use of diagnostic aids and longer professional training. The checklist toolkit assessments resulted in some participants being wrongly assessed as eligible for medical abortion which is an area of concern. Further research is needed to streamline the components of the tool, explore optimal duration and content of training for community health workers, and test feasibility and acceptability.
Citation: Johnston HB, Ganatra B, Nguyen MH, Habib N, Afework MF, Harries J, et al. (2016) Accuracy of Assessment of Eligibility for Early Medical Abortion by Community Health Workers in Ethiopia, India and South Africa. PLoS ONE 11(1): e0146305. https://doi.org/10.1371/journal.pone.0146305
Editor: Kristina Gemzell-Danielsson, Karolinska Institutet, SWEDEN
Received: June 11, 2015; Accepted: December 15, 2015; Published: January 5, 2016
Copyright: © 2016 Johnston et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: All relevant data are within the paper and its Supporting Information files. Further information can also be requested from the WHO by writing to the corresponding author.
Funding: The UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP), Department of Reproductive Health and Research (RHR), World Health Organization (BG). The Department coordinated technical review of study protocols by an external peer group and provided support for the publication of study results.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Community health workers (CHWs) are increasingly important frontline providers of health information and services in maternal health, contraceptive services, HIV and tuberculosis management and in community level diagnosis of neonatal and childhood illness [1–7].
In many countries the challenges associated with accessing providers of safe abortion care contribute to unsafe abortion and related morbidity and mortality. In 2008 an estimated 22 million pregnancies were terminated unsafely, 98% of which occurred in resource-poor countries [8] where health care worker shortages are most prominent. While the safety and effectiveness of facility-based non-physician providers of early abortion care using vacuum aspiration or medical abortion has been documented [9–18] the supportive roles of community-based providers have received less attention.
Medical abortion in early pregnancy can be provided at the primary care level and on an outpatient basis. Medical contraindications are limited to a few uncommon conditions like chronic or acute adrenal or hepatic failure, inherited porphyria and allergy to mifepristone or misoprostol. Additionally, clinical judgment may be needed for women with long term corticosteroid use, bleeding disorders, severe anemia, pre-existing heart disease or cardiovascular risk factors [19]. Screening for eligibility is usually done by the clinician providing the medical abortion but potentially could be conducted by other health workers in the clinic or even outside of a health facility. However, to date the tools and approaches for this have not been fully developed or validated.
We conducted a prospective clinic-based diagnostic accuracy study with the dual aim of evaluating the validity of a checklist toolkit to assess a woman's eligibility for early medical abortion (≤ 63 days); and assessing the performance of CHWs in using this checklist toolkit.
Methods
Study context and participants
The study was conducted in Ethiopia, India and South Africa, countries where medical abortion with mifepristone and misoprostol is legal and available but unsafe abortion remains a serious health issue [20–24] and CHWs are actively involved in providing health care. In Ethiopia health extension workers are deployed at health posts that provide primary health services. In India accredited social health activists (ASHAs) provide basic health care including pregnancy tests, maternal health counselling and referral, and contraceptive counselling, supply and referral at the village-level. In South Africa, where the National Department of Health’s CHW strategy supports CHWs through NGOs [25], such CHWs typically provide counselling and referral for HIV/AIDS and tuberculosis care.
As this was an initial validation study it was done in clinic rather than community settings. Participants were recruited from one government and two non-governmental organization (NGO) clinics in urban Adama and Assela in Ethiopia; at two urban and three rural NGO clinics in Udaipur, Rajasthan in India; and at five urban NGO clinics in Durban and Cape Town in South Africa. All women who presented to the study clinics for an elective abortion during the study period were invited to participate if they were at least 18 years old, willing to provide informed consent, and able to understand the nature of the study, questions, and instructions given by the research assistant. Willingness to participate in the study was documented by participant signatures or thumbprints on consent forms developed specifically for each study site. The study protocol, including consent procedures, was approved by the Ethics Review Committee at the World Health Organization as well as the Ethiopian Public Health Association Internal Scientific and Technical Review Committee on 4 January 2013 (EPHA/OG/575/13), the Institutional Ethics Committee, Action Research & Training for Health, Udaipur India on 27 November 2012, and the University of Cape Town Faculty of Health Sciences Human Research Ethics Committee on 23 November 2011 (HREC REF: 483/2011).
Clinical assessors in both Ethiopia (N = 8) and South Africa (N = 10) were either health officers or nurses with 36–48 months of professional training. Since such providers are not allowed to provide abortion care in India, clinical assessors in that setting were obstetrician-gynecologists (N = 7). CHW assessors in Ethiopia (N = 10) were health extension workers with a minimum of 10 years of basic education and 12 months of public health training; in India (N = 12) they were ASHAs with a minimum of eight years of basic education and 20 days of public health training or village health workers with a minimum of five years of basic education and 28 days of public health education; and in South Africa (N = 7) they were community-based educators with a minimum of 12 years of basic education and 20 days of public health training.
Study procedures
The checklist toolkit included a urine pregnancy test, a gestational age wheel, and a checklist [S1 Fig] of screening questions related to result of urine pregnancy test, date of last menstrual period, result of gestational age wheel assessment, and seven questions related to possible contraindications to medical abortion (ectopic pregnancy, bleeding disorders, history of inherited porphyria, serious medical illness, long term medication and allergy to medical abortion drugs). The checklist and gestational age wheel were translated into local languages in Ethiopia and India. Additionally, in Ethiopia the gestational age wheel was adapted to reflect the Ethiopian calendar.
CHWs underwent study-specific training of two to four days (training duration was longer in India where CHWs had less prior formal training) in basic reproductive physiology, medical abortion and implementation of the checklist toolkit. Competency was judged on training posttests. Clinician training was limited to review of study methods and tools.
To be screened-in as eligible for medical abortion the study participant had to have a positive pregnancy test, be within 63 days (nine weeks) gestation as calculated with the gestational age wheel and have no positive responses on the screening questions.
Study participants were initially assessed by a CHW and subsequently by the clinician using identical checklists. Results were classified as eligible, ineligible, or labelled inconclusive if assessors were unable to reach a decision based on the checklist information. After clinicians completed the checklist they assessed eligibility using standard clinical procedures including aa bimanual or ultrasound examination to assess gestational age. This final assessment was considered the reference standard exam, and study participants were classified as eligible or ineligible for medical abortion. Clinicians completed the checklist and placed results in an envelope prior to conducting the reference standard exam.
Data collection forms were reviewed for accuracy and completeness by study staff in-country and at the World Health Organization. Data were double-entered into OpenClinica web-based data management system version 3.1.3 (Akaza Research, Waltham, MA, USA). Validation checks and periodic review of missing values, outliers, inconsistencies, and other errors were conducted. All statistical analyses were done using SAS/STAT ® software (SAS Institute. 2011. The SAS system for Windows. Release 9.3. SAS Inst., Cary, NC, USA).
Study design, sample size and analytical methods
The sample size of 211 participants at each site was calculated with the assumption that of the women seeking care for abortion, 60% would be eligible for medical abortion. This sample size met the objectives of generating a two-sided 90% confidence interval for sensitivity with a width of 15% assuming a sensitivity score close to 60%, and a two-sided 90% confidence interval for specificity with a width of 15%, assuming a specificity score of 80%.
Baseline characteristics for participants presenting for medical abortion were described, as were characteristics of participating CHWs and clinicians.
We compared results of checklist assessments by clinicians and by CHWs with results of the reference standard assessment by computing statistics for overall accuracy, sensitivity, specificity, positive and negative predictive values, and positive and negative likelihood ratios.
Study recruitment took place between November 2012–February 2014. Project implementation in each country was timed according to partner availability and project approvals.
Results
CHWs in each country scored similarly on training posttests which measured recall of the didactic material presented in the training (Ethiopia mean: 88% (SD 0·17), India mean: 82% (SD 0·25), South Africa mean: 89% (SD 0·18)).
Ninety-five percent (n = 217) of women coming to the study sites in Ethiopia agreed to participate, 98% in India (n = 258), and 92% in South Africa (n = 236) (Fig 1). Most were in the 20–29 year age group (Table 1). In n India half of participants had no formal education; Sixty percent of participants in Ethiopia and 98% in South Africa were educated beyond class nine.
Reference standard exams were conducted on 206 women in Ethiopia (three forms were missing), 254 women in India (two forms were missing) and 233 women in South Africa. Of these, 163 women (80%) were found to be eligible for medical abortion according to the reference standard exam in Ethiopia, 197 in India (78%) and 153 (66%) in South Africa (Fig 1). Cases whose status was determined inconclusive in the checklist assessment and those with reference standard exam eligibility status missing were excluded from the analysis.
The most common reason for women being determined as ineligible for medical abortion in the reference standard exam was pregnancy >63 days. This was the reason in 63% of the ineligible cases in Ethiopia (25/40); 64% of the ineligible cases in India (35/55) and 89% of the ineligible cases in South Africa (71/80). In Ethiopia 14, India eight, and South Africa two study participants were found to be non-pregnant in the reference standard exam. In India five and in South Africa two study participants required further assessment even after the reference exam. In India three women had already started taking medical abortion tablets prior to coming to the clinic. One study participant in South Africa was suspected to have an ectopic pregnancy; one in Ethiopia was suspected to have a bleeding disorder; four in India and four in South Africa were labelled ineligible for miscellaneous clinical reasons.
Checklist toolkit as used by clinicians
Accuracy of the checklist toolkit was over 90% and negative likelihood ratios were less than 0.1 at all three sites. Positive likelihood ratios were 4.3 in Ethiopia, 5.8 in India and 6.3 in South Africa (Table 2A).
Clinician checklist assessments of pregnancy duration as ≤63 days or >63 days matched the reference standard in 96% of cases in Ethiopia, 88% in India and 95% in South Africa. Study participants were incorrectly assessed as ≤63 days by clinicians with the checklist,in eight cases in Ethiopia (five of these were between 63-≤70 days, two were 70–84 days, and one 143 days), 17 cases in India (nine were between 63-≤70 days, six were 70–84 days, and two 105 days); and seven cases in South Africa (one was between 63-≤70 days, four were 70–84 days, one 91 days and one 112 days).
In a small number of cases (one in Ethiopia, four in India, seven in South Africa), clinician assessors could not reach a decision based on the eligibility checklist, usually because of uncertainty of the date of last menstrual period.
The one study participant with a suspected ectopic pregnancy was not identified by the clinician using the checklist toolkit.
Checklist toolkit as used by community health workers
Accuracy of CHW use of the checklist toolkit was 92% in Ethiopia, 80% in India and 77% in South Africa. The positive likelihood ratio was 5.9 in Ethiopia and 2.0 in both India and South Africa. The negative likelihood ratio was 0.08 in Ethiopia, 0.25 in India and 0.22 in South Africa (Table 2B).
CHWs did not reach a conclusion using the eligibility checklist toolkit for eight women in Ethiopia and South Africa and two in India (Fig 2), usually because of uncertain date of last menstrual period.
CHW categorization of duration of gestation was in agreement with the reference standard in 95% of cases in Ethiopia, 83% in India, and 90% in South Africa. Study participants were incorrectly assessed as ≤63 days by CHWs with the checklist toolkit in six cases in Ethiopia, 29 cases in India and 19 cases in South Africa. In Ethiopia five of these six cases were assessed in the clinician physical exam as 63-≤70 days and one 126 days; in India 11 of the 29 cases were 63-≤70 days, four 77 days, six 84 days, seven between 98–126 days, and one 168 days; and in South Africa six of the 19 cases were 63-≤70 days, five 77 days, four 84 days, three between 91–112 days and one 168 days.
The study participant with a suspected ectopic pregnancy was not identified by the CHW with the checklist toolkit.
Discussion
This is the first study on CHW use of a checklist toolkit to assess women’s eligibility for medical abortion.
Clinician use of the toolkit yielded negative likelihood ratios of <0.1 in all three countries. Results were similar with Ethiopian CHW assessors, but more modest in India and South Africa. A negative likelihood ratio of <0.1 is a good indicator that the likelihood of a woman who is eligible for medical abortion being wrongly classified as ineligible is low.
However, clinically, it is even more important to ensure that women who are over the gestational age cut off or have other contraindications are not incorrectly labelled as eligible. Results were more modest in this regard with positive likelihood ratios of 4–6 with clinician use of the checklist. With CHWs using the checklist, positive likelihood ratios were modest in Ethiopia but less so in India and South Africa.
Mifepristone and misoprostol are effective even beyond 63 days and recent research suggests similar efficacy using the current regimens to 70 days [26–27]. However a small number (n = 17) of study participants assessed as ≤63 days by clinicians using the checklist toolkit were >70 days gestation according to physical exam results, which is of concern. Women’s self-recall of days since most recent menstrual period is a reasonable proxy for gestational age assessment especially at <63 days [28–29]. Yet in this study, not all participants were able to provide an accurate first date of their last menstrual period and this may have resulted in these discrepancies.
Additionally, at all three sites, CHWs made errors in the use of the gestational age wheel and the recording of its results. This signifies the need for further research in identifying optimum duration and content of training in order to improve the accuracy of CHW assessments.
Overall, the accuracy of CHW checklist toolkit results was higher in Ethiopia than in India and South Africa. This may have been because health extension workers in Ethiopia had higher levels of pre study training and more experience with use of diagnostic aids. As with any new technique, there is a learning curve and results may have improved in India and South Africa over time; the study design however did not allow us to test for this.
The one woman suspected to have an ectopic pregnancy on the reference standard exam was not identified as such through the checklist toolkit. Ectopic pregnancy is rare and even if a woman with an ectopic pregnancy receives medical abortion, it would be ineffective but it would not worsen the condition. Nevertheless, this represents a missed opportunity for early detection. Further iterations of the toolkit could refine screening questions relating to identifying a possible ectopic pregnancy. Very few women were identified as having other medical contraindications which is not surprising given that absolute contraindications to medical abortion are few and the conditions they represent are rare.
One study limitation is that the same clinician who completed the checklist toolkit assessment conducted the reference standard physical exam. We emphasized to clinicians that checklist results must not be altered based on results of the reference standard physical exam but the possibility that this occurred in some instances cannot be ruled out.
The results show that, as used by clinicians and by CHWs in Ethiopia, the checklist toolkit is reasonably accurate and works well for ruling out ineligible cases. It is less effective in ensuring that women are not incorrectly ruled in to receive medical abortion when they should not be. The current version thus cannot replace a second level clinical assessment of eligibility but could be used as a community level screening tool to facilitate appropriate referrals and timely triage. That CHWs with minimal schooling or prior experience with diagnostic aids were also able to use the toolkit, suggests the potential of developing this approach further.
Conclusion
Simple tools to help community health workers assess eligibility for early medical abortion is a promising approach that has the potential to decentralize access to safe care and overcome shortages of physician providers. Further research is needed to refine the checklist toolkit and explore the feasibility and acceptability within community settings.
Supporting Information
S1 Fig. Medical abortion eligibility checklist tool.
https://doi.org/10.1371/journal.pone.0146305.s001
(PDF)
Acknowledgments
Hailu Yeneneh Lema, co-author of this paper passed away in December 2015. We dedicate the paper to him in grateful recognition of his spirited and manifold contributions to this study and his tireless efforts in supporting public health in Ethiopia. We thank Marie Stopes International in South Africa and in Ethiopia, Mosaic in Cape Town, South Africa and the Parivar Seva Sanstha, Udaipur, India. We thank Kathryn Andersen, Mary Fjersted, Haile Gebreselassie, Thoai Ngo and I K Warriner for their inputs in developing the study.
Author Contributions
Conceived and designed the experiments: BG MHN JH KI HBJ. Performed the experiments: HBJ MHN MFA JH KI JM HYL DC SS BG. Analyzed the data: NH BG MHN HBJ. Wrote the paper: BG HBJ NH MHN KI MFA JH JM HYL DC.
References
- 1. Baqui AH, Arifeen SE, Rosen HE, Mannan I, Rahman SM, Al-Mahmud AB, et al. Community-based validation of assessment of newborn illnesses by trained community health workers in Sylhet district of Bangladesh. Trop Med Int Health. 2009; 14: 1448–1456. pmid:19807901
- 2. Celletti F, Wright A, Palen J, Frehywot S, Markus A, Greenberg A, et al. Can the deployment of community health workers for the delivery of HIV services represent an effective and sustainable response to health workforce shortages? Results of a multicountry study. AIDS. 2010; 24: S45–57.
- 3. Heunis JC, Wouters E, Norton WE, Engelbrecht MC, Kigozi NG, Sharma A et al. Patient- and delivery-level factors related to acceptance of HIV counseling and testing services among tuberculosis patients in South Africa: a qualitative study with community health workers and program managers. Implement Sci. 2011; 6.
- 4. Lewin S, Munabi-Babigumira S, Glenton C, Daniels K, Bosch-Capblanch X, van Wyk BE, et al. Lay health workers in primary and community health care for maternal and child health and the management of infectious diseases. Cochrane Database Syst Rev. 2010; 3.
- 5. Malarcher S, Meirik O, Lebetkin E, Shah I, Spieler J, Stanback J. Provision of DMPA by community health workers: what the evidence shows. Contraception. 2011; 83: 495–503. pmid:21570545
- 6. Darmstadt GL, Baqui AH, Choi Y, Bari S, Rahman SM, Mannan I, et al. Validation of community health workers' assessment of neonatal illness in rural Bangladesh. Bull World Health Organ. 2009; 87: 12–19. pmid:19197400
- 7.
World Health Organization. WHO recommendations: optimizing health worker roles to improve access to key maternal and newborn health interventions through task shifting. Geneva: World Health Organization; 2012.
- 8.
World Health Organization. Unsafe abortion: Global and regional estimates of the incidence of unsafe abortion and associated mortality in 2008; 6th ed. Geneva: World Health Organization; 2011.
- 9. Bhatia S, Faruque AS, Chakraborty J. Assessing menstrual regulation performed by paramedics in rural Bangladesh. Stud Fam Plann. 1980; 11: 213–218 pmid:7414673
- 10. Dickson-Tetteh K, Billings DL. 2002. Abortion care services provided by registered midwives in South Africa. Int Fam Plan Perspect. 2002; 28: 144–150.
- 11. Freedman MA, Jillson DA, Coffin RR, Novick LF. Comparison of complication rates in first trimester abortions performed by physician assistants and physicians. Am J Public Health. 1986; 76: 550–554. pmid:2870654
- 12. Goldman MB, Occhiuto JS, Peterson LE, Zapka JG, Palmer RH. Physician assistants as providers of surgically induced abortion services. Am J Public Health. 2004; 94: 1352–1357. pmid:15284043
- 13. Jejeebhoy S, Kalyanwala S, Zavier A, Kumar R, Mundle S, Tank J et al. Can nurses perform manual vacuum aspiration (MVA) as safely and effectively as physicians? Evidence from India. Contraception. 2011; 84: 615–621. pmid:22078191
- 14. Jejeebhoy SJ, Kalyanwala S, Mundle S, Tank J, Zavier AJF, Kumar R, et al. Feasibility of expanding the medication abortion provider base in India to include ayurvedic physicians and nurses. Int Perspect Sex Reprod Health. 2012; 38:133–142. pmid:23018135
- 15. Scotti RJ, Karman HL. 1976. Menstrual regulation and early pregnancy termination performed by paraprofessionals under medical supervision. Contraception. 14: 367–374. pmid:975822
- 16. Warriner IK, Meirik O, Hoffman M, Morroni C, Harries J, Nguyen TMH, et al. Rates of complication in first-trimester manual vacuum aspiration abortion done by doctors and mid-level providers in South Africa and Vietnam: a randomised controlled equivalence trial. Lancet. 2006; 368:1965–1972. pmid:17141703
- 17. Warriner IK, Wang D, Nguyen TMH, Thapa K, Tamang A, Shah I, et al. Can midlevel health-care providers administer early medical abortion as safely and effectively as doctors? A randomised controlled equivalence trial in Nepal. Lancet. 2011; 377: 1155–1161. pmid:21458058
- 18. Weitz TA, Taylor D, Desai S, Upadhyay UD, Waldman J, Battistelli MF, et al. Safety of aspiration abortion performed by nurse practitioners, certified nurse midwives, and physician assistants under a California legal waiver. Am J Public Health. 2013; 103: 454–462. pmid:23327244
- 19.
World Health Organization. Safe abortion: technical and policy guidance for health systems. 2nd ed. Geneva: World Health Organization; 2012.
- 20. Abdella A, Fetters T, Benson J, Pearson E, Gebrehiwot Y, Andersen K, et al. Meeting the need for safe abortion care in Ethiopia: Results of a national assessment in 2008. Glob Public Health. 2013; 8: 417–434. pmid:23590804
- 21.
Sample Registration System. Maternal mortality in India: trends, causes and risk factors. New Delhi: Registrar General of India; 2006.
- 22.
National Department of Health (South Africa). Saving mothers 2008–2010: Fifth report on the confidential enquiries into maternal deaths in South Africa. Department of Health, South Africa; 2012.
- 23. Constant D, Grossman D, Lince N, Harries J. Self-induction of abortion among women accessing second-trimester abortion services in the public sector, Western Cape, South Africa: An exploratory study. S Afr Med J. 2014; 104: 302–305. pmid:25118559
- 24. Basu JK and Basu D. Morbidity from unsafe termination of pregnancy in South Africa. J Obstet Gynaecol. 2013; 33: 605–608. pmid:23919860
- 25.
National Department of Health (South Africa). Re-engineering Primary Health Care in South Africa, Discussion document. National Department of Health; 2010. Available: http://indicators.hst.org.za/indicators/DoH/PHC_Reengineering241110.docx.
- 26. Sanhueza Smith P, Peña M, Dzuba IG, Martinez ML, Peraza AG, Bousiéguez M, et al. Safety, efficacy and acceptability of outpatient mifepristone-misoprostol medical abortion through 70 days since last menstrual period in public sector facilities in Mexico City. Reprod Health Matters. 2015; 22(Suppl 1):75–82
- 27. Abbas D, Chong E, Raymond EG. Outpatient medical abortion is safe and effective through 70 days gestation. Contraception. 2015; 92: 197–199. pmid:26118638
- 28. Schonberg D, Wang L, Bennett A, Gold M, Jackson E. The accuracy of using last menstrual period to determine gestational age for first trimester medication abortion: a systematic review. Contraception. 2014; 90: 480–487. pmid:25152258
- 29. Raymond EG, Bracken H. Early medical abortion without prior ultrasound. Contraception. 2015; 92: 212–214. pmid:25916975