Background
Methods
Adaptation and familiarization with the tool
Structure of the tool
Scoring of each component of care
Study sites
The assessment process
Data analysis
Results
Area of care | Standards assessed | Number (n) and proportion (%) of facilities where standards are met (scoring 4 or 5) | Mean score | Mean score | ||||||
---|---|---|---|---|---|---|---|---|---|---|
District 1 n = 7 (%) | District 2 n = 7 (%) | District 3 n = 7 (%) | District 4 n = 5(%) | District 5 n = 5(%) | Total N = 31 (%) | Across 5 districts (SD) | Level of care | |||
Module A: Infrastructure | CEmOC | BEmOC | ||||||||
A2. Infrastructure | Availability of electricity, back-up power supply, running water, soap/disinfectant, sharps disposal, fridge for drugs, oxygen source, complaints/ suggestions box | 3 (42.8) | 5 (71.4) | 4 (57.1) | 3 (60) | nda
| 15 (48.3) | 3.8 (0.83) | 4.3 (0.6) | 3.8 (0.85) |
A5. Health information & record keeping | Health info: reporting forms, monthly reports of OPD visits, computer based system for medical indicators, periodic facility review of stats and indicators Records: clarity, legibility, availability, access | 2 (28.6) | 5 (71.4) | 4 (57.1) | 2 (40) | nd | 13 (41.9) | 3.8 (0.79) | 4.0 (0.8) | 3.7 (0.81) |
A6. Essential drugs & blood products | List of essential drugs checked for presence, regular availability, expiry Blood bank at facility, blood units available | 2 (28.6) | 6 (85.7) | 1 (14.3) | 3 (60) | 2 (40) | 14 (45.2) | 3.8 (0.73) | 4.3 (0.5) | 3.7 (0.75) |
A7. Laboratory | Various lab investigations observed: if test available, results readily available, average time to get results | 3 (42.8) | 3 (42.8) | 2 (28.6) | 4 (80) | nd | 12 (38.7) | 3.3 (1.06) | 4.2 (0.4) | 3.1 (1.06) |
A8. Health facility policies | Policies checked for availability: hospital fees, free drugs, services for emergency cases, community participation, infection control, hand hygiene, in service training, quality of care, KMC, essential drugs list | 2 (28.6) | 2 (28.6) | 5 (71.4) | 4 (80) | nd | 13 (41.9) | 3.5 (0.74) | 4.0 (0.0) | 3.3 (0.79) |
A9. Referral | Functioning vehicle with fuel routinely available for emergency referral use, referral transport provided free of charge, clear criteria for referral, communication with referral facility, referral time | 1 (14.3) | 7 (100) | 4 (57.1) | 4 (80) | nd | 16 (51.6) | 3.7 (1.13) | 4.4 (0.9) | 3.6 (1.15) |
A10. Guidelines/ audit | Guideline availability for normal labour & delivery, emergency conditions for mothers, newborns and children, routine care for newborns, management of childhood illnesses, recent obstetric, neonatal and paediatric textbook Conduct of audits to review deaths and complications (MDSR, neonatal, perinatal, paediatric), functioning of teams, recommendations discussed and implemented, periodic meetings to discuss issues and revise protocols | 0 (0) | 3 (42.8) | 2 (28.6) | 0 (0) | nd | 5 (16.1) | 2.6 (1.11) | 3.5 (0.6) | 2.4 (1.12) |
Module B: Maternal Care | CEmOC | BEmOC | ||||||||
B1. Emergency obstetric care | Patient flow, layout and structure of emergency care | 0 (0) | 6 (85.7) | 3 (42.9) | 3 (60) | 1 (20) | 13 (41.9) | 3.4 (0.77) | 3.2 (0.8) | 3.5 (0.79) |
B2. Maternity wards | Ward staffing, hygiene & accident prevention, attention to most seriously ill | 0 (0) | 6 (85.7) | 2 (28.6) | 2 (40) | 1 (20) | 11 (35.4) | 3.2 (0.77) | 3.0 (0.9) | 3.3 (0.75) |
B3. Infection control & supportive cares | Infection control: hand hygiene, use of gloves, infection control practices | 1 (14.3) | 5 (71.4) | 0 (0) | 0 (0) | 0 (0) | 6 (19.3) | 3 (0.74) | 3.2 (0.8) | 3.0 (0.75) |
Supportive care: nutritional needs of admitted patients, IV fluids, drug treatment, blood transfusion | 3 (42.9) | 5 (71.4) | 4 (57.1) | 5 (100) | 5 (100) | 22 (70.9) | 4 (0.62) | 3.8 (0.4) | 4.1 (0.65) | |
B4. Essential drugs, equipment and supplies | Essential drugs: checked for presence, regular availability, expiry: water, normal saline. Ringers lactate, hydralazine/nifedipine, methyldopa, oxytocin, misoprostol, ergometrine, calcium gluconate, injectable MgSO4, diazepam, ampicillin/amoxycilin, benzylpenicillin, gentamycin, metronidazole, lignocaine 2% or 1%, halothane/isoflurane, ketamine | 1 (14.3) | 6 (85.7) | 1 (14.3) | 2 (40) | 2 (40) | 12 (38.7) | 3.5 (0.87) | 4.0 (1.0) | 3.4 (0.85) |
Equipment and supplies: checked for presence, safety, hygiene, good working order: lighting, exam light, wall clock, delivery pack, partographs, heat lamp, CS set, oxygen cylinder, concentrator, supply & flow metres and equipment for administration of oxygen, respiratory support bags, bags and masks (adult and neonatal size), anaesthetic equipment, thermometer, sterile gloves, sterile gauze, foetal stethoscope, stethoscope, BP machine, infusion sets & pumps, IV catheters, urinary catheter & bags, syringes, needles & cannulas, suture set, sharps container, vacuum extractor, vacuum aspirator, delivery, regular & operating beds, neonatal equipment | 1 (14.3) | 4 (57.1) | 3 (42.9) | 4 (80) | 3 (60) | 15 (48.4) | 3.4 (0.73) | 3.8 (0.4) | 3.4 (0.77) | |
B5. Antepartum care | Eclampsia and pre-eclampsia: diagnosis, management of mild pre-eclampsia, management of severe pre-eclampsia and eclampsia | 1 (14.3) | 2 (28.6) | 4 (57.1) | 5 (100) | 0 (0) | 12 (38.7) | 3.3 (0.77) | 3.4 (0.5) | 3.3 (0.82) |
Infections during pregnancy: Antibiotic use inf pregnancy, screening and management of HIV in pregnancy, management of severe malaria in pregnancy | 5 (71.4) | 4 (57.1) | 3 (42.9) | 5 (100) | 5 (100) | 22 (71) | 4 (0.54) | 4.2 (0.4) | 4.0 (0.58) | |
B5. Intrapartum & postpartum care | Normal labour and vaginal delivery: delivery conditions, supportive care throughout labour, care during first, second and third stage of labour, care of mother after delivery | 1 (14.3) | 5 (71.4) | 4 (57.1) | 1 (20) | 2 (40) | 13 (41.9) | 3.5 (0.70) | 3.0 (0.0) | 3.6 (0.73) |
Preterm labour: management of pre-term labour, antenatal administration of corticosteroids | 2 (28.6) | 3 (42.9) | 1 (14.3) | 1 (20) | 4 (80) | 11 (35.5) | 3 (1.33) | 4.3 (0.5) | 2.6 (1.23) | |
Caesarean section (Assessed at 6 EmONC facilities only): emergency CS, timing of CS and informed consent, indications and policies to reduce inappropriate CS, procedures related to CS, immediate post-CS care, care after 24 h and discharge after CS | 1 (50) | nd | 1 (100) | 1 (100) | 1 (100) | 4 (66.6) | 3.6 (0.89) | 3.5 (1.0) | nd | |
PPH guidelines available and known by SBAs, adequate equipment, drugs and personnel available, uterine tonus controlled after delivery, reliable way of measuring blood loss in delivery room and early puerperium, blood can be obtained without delay 24 h a day, written protocols available, known and used by staff for notification of on calls, treatment and resuscitation, uterine atony, and refractory haemorrhages after treatment | 0 (0) | 7 (100) | 1 (14.3) | 3 (60) | 3 (60) | 14 (45.2) | 3.5 (0.97) | 3.4 (0.5) | 3.6 (1.06) | |
Unsatisfactory progress of labour: diagnosis of labour disorders, prolonged active phase management | 6 (85.7) | 7 (100) | 3 (42.9) | 5 (100) | 5 (100) | 26 (83.9) | 4.2 (0.57) | 4.2 (0.4) | 4.2 (0.61) | |
Monitoring & follow up of admitted patients: monitoring of individual progress, reassessment by midwives, reassessment by clinicians, follow up | 1 (14.3) | 6 (85.7) | 3 (42.9) | 4 (80) | 3 (60) | 17 (54.8) | 3.6 (1.0) | 3.5 (0.8) | 3.6 (1.09) | |
Module C: Neonatal Care | CEmOC | BEmOC | ||||||||
C1. Nursery facilities | Ward staffing, hygiene & accident prevention | 0 (0) | 4 (57.1) | 5 (71.4) | 3 (60) | 1 (20) | 13 (41.9) | 3.2 (1.19) | 3.5 (0.8) | 3.1 (1.3) |
C2. Infection control & supportive care | Infection control and hygiene, use of gloves, infection control practices | 2 (28.6) | 4 (57.1) | 3 (42.9) | 0 (0) | 0 (0) | 9 (29) | 3.3 (0.93) | 3.2 (0.4) | 3.6 (1.06) |
Supportive care: IV fluids use, drug treatment, central venous cannulation & umbilical catheterization, blood transfusion | 0 (0) | 2 (28.6) | 5 (71.4) | 2 (40) | 0 (0) | 9 (29) | 3.5 (0.83) | 3.7 (1.2) | 3.5 (0.65) | |
C3. Essential drugs, equipment & supplies | Essential drugs checked for presence, regular availability, expiry: antibiotics, antivirals and antifungals, CNS drugs, respiratory drugs, miscellaneous including chlorhexidine, paracetamol suspension, nevirapine suspension, vitamins & minerals | 3 (42.9) | 3 (42.9) | 5 (71.4) | 2 (40) | 1 (20) | 14 (45.2) | 3.4 (1.0) | 4.0 (0.0) | 3.2 (1.17) |
Equipment and supplies checked for presence, safety, hygiene, good working order: incubators, radiant warmers, baby cots, phototherapy lamps, ambu bag, oxygen supply, concentrator & nasal prongs, appropriate sized face masks, CPAP systems, multifunction monitors, pulse-oximeters, breast pumps, nasogastric tubes, microdroppers, exchange transfusion, stethoscope, glucometers and sticks, suction apparatus, thermometers, electric weighing scale, linen | 1 (14.3) | 2 (28.6) | 2 (28.6) | 2 (40) | 0 (0) | 7 (22.6) | 2.9 (0.95) | 3.3 (0.8) | 2.8 (0.98) | |
C4. Routine care | Neonatal resuscitation, assessment & immediate care, screening, prevention & management of infectious diseases, early & exclusive breastfeeding, monitoring before discharge, counselling for mothers | 1 (14.3) | 2 (28.6) | 7 (100) | 5 (100) | 0 (0) | 15 (48.4) | 3.7 (0.70) | 3.5 (1.0) | 3.81 (0.54) |
C5. Management of sick newborn | Management of preterm and LBW babies, neonatal sepsis, recognition & treatment of jaundice, management of convulsions, feeding for sick neonates | 0 (0) | 1 (14.3) | 3 (42.9) | 1 (14.9) | 0 (0) | 5 (16.1) | 3.2 (0.93) | 3.2 (1.2) | 3.2 (0.75) |
Module D: Paediatric Care | CEmOC | BEmOC | ||||||||
D1. Emergency paediatric care | Patient flow, emergency care staffing, layout & structure of emergency care, case management of emergency conditions | 0 (0) | 4 (57.1) | 1 (14.9) | 0 (0) | 3 (60) | 8 (25.8) | 3.2 (0.69) | 3.4 (0.9) | 3.1 (0.64) |
D2. Paediatric wards | Staffing level, hygiene & accident prevention, attention to most seriously ill patients | 0 (0) | 1 (100) | 1 (100) | nd | nd | 2 (33.3) | 3.2 (0.96) | 3.3 (1.0) | nd |
D3. Infection control & supportive care | Infection control: Hand hygiene, use of gloves, practices for infection control | 0 (0) | 0 (0) | 5 (71.4) | 0 (0) | 3 (60) | 8 (25.8) | 3.2 (0.60) | 3.4 (0.5) | 3.2 (0.62) |
Supportive care: Nutritional needs, use of IV fluids, drug treatment, blood transfusion | 5 (71.4) | 4 (57.1) | 7 (100) | 4 (80) | 2 (40) | 22 (71) | 4.2 (0.66) | 2.6 (0.9) | 2.8 (1.11) | |
D4. Essential drugs, equipment & supplies | Essential drugs checked for presence, regular availability, expiry: glucose (30–50%, 10%, 5%), saline, ringers lactate, half strength darrows, RESOMOL, adrenaline, corticosteroids IV or oral, furosemide IV, first line anticonvulsant, antibiotics, anti-malarials, paracetamol (oral, suppository), vitamin K, ORS, zinc sulphate | 3 (42.9) | 3 (42.9) | 3 (42.9) | 2 (40) | 3 (60) | 11 (35.5) | 3.5 (0.76) | 3.4 (0.5) | 3.5 (0.81) |
Equipment and supplies checked for presence, safety, hygiene, good working order: resuscitation table, torch, autoscope, scales for children, measuring board, stethoscope, thermometer, heat source, oxygen cylinder, concentrator & supply, oxygen flow meter, supplies for oxygen administration, resuscitation bags, IV sets, infusion pumps, burettes, cannulas/butterflies paediatric size, feeding syringes, NG tubes | 0 (0) | 1 (14.9) | 2 (28.6) | 0 (0) | 2 (40) | 5 (16.1) | 2.7 (1.1) | 2.6 (0.9) | 2.8 (1.11) | |
D5. Case management of common diseases | Cough/difficult breathing: assessment & treatment of pneumonia, administration of antibiotics, administration of oxygen therapy, use of chest x-ray, management of wheezing, TB treatment | 1 (14.9) | 1 (14.9) | 2 (28.6) | 4 (80) | 2 (40) | 10 (32.3) | 3.4 (0.91) | 3.2 (0.4) | 3.5 (1.0) |
Diarrhoea: assessment of dehydration, management of dehydration & diarrhoea, antibiotic use, continued feeding | 3 (42.9) | 2 (28.6) | 5 (71.4) | 4 (80) | 3 (60) | 17 (54.8) | 3.7 (0.68) | 3.4 (0.5) | 3.8 (0.7) | |
Fever: differential diagnosis & investigations, diagnosis & management of meningitis, diagnosis & management of severe/complicated malaria, diagnosis & management of measles, diagnosis & management of other severe febrile conditions | 1 (14.9) | 3 (42.9) | 7 (100) | 4 (80) | 4 (80) | 19 (61.3) | 3.8 (0.62) | 3.6 (0.5) | 3.9 (0.64) | |
Malnutrition: assessment of nutritional status, management of infection & micronutrient supplementation, management of dehydration, prevention & management of hypoglycaemia & hypothermia, feeding of severely malnourished children, management of associated conditions & supportive care | 4 (57.1) | 0 (0) | 6 (85.7) | 4 (80) | 4 (80) | 18 (58) | 4 (0.59) | 3.8 (0.4) | 4.0 (0.63) | |
HIV/AIDS: counselling & diagnosis of paediatric HIV/AIDS, ARV treatment and monitoring, opportunistic infections & supportive care, supportive care & follow up of HIV infected children | 3 (42.9) | 3 (42.9) | 6 (85.7) | 4 (80) | 3 (60) | 19 (61.3) | 4.3 (0.66) | 3.7 (0.6) | 4.4 (0.62) | |
D5. Monitoring & follow up of admitted children | Monitoring of individual progress, monitoring by nurses, reassessment by clinicians, follow up | 1 (14.9) | 0 (0) | 6 (85.7) | 2 (40) | 2 (40) | 11 (35.5) | 3.8 (1.0) | 2.6 (0.5) | 4.2 (0.86) |
D6. Paediatric surgery & rehabilitation | Paediatric size anaesthesia-equipment, pre-, intra- and post-operative care, rehabilitation | nd | nd | nd | nd | 0 (0) | 1 (16.7) | 3.5 (0.70) | 3.5 (0.7) | nd |
Results by level of care
Key emerging issues
Discussion
Strengths and limitations of the new WHO tool
Shorten the tool | Because it is so comprehensive, the tool can be adapted to suit local disease and population profiles and used to assess different levels of care. However, the adapted assessment tool applied in Malawi is over 200 pages long. A shorter tool, focused on priority areas of care, may be more appropriate for use in busy clinical environments. |
Combine the CEmOC and BEmOC tools | There are currently two versions of the tool and the only difference is that the CEmOC tool assesses management of Caesarean section (including pre-, intra- and post-operative procedures), paediatric wards and paediatric surgery and rehabilitation. It should be possible to combine the tools into one, with separate modules for assessing the standards relevant to CEmOC facilities only. |
Streamline data sources and revise instructions for data collection | The data sources could be streamlined, and clearer instructions for conducting the various components of the assessment provided. The number of data collection methods and lack of clear guidance on when and how they should be used is potentially confusing to assessors who have limited time to complete facility visits. |
Simplify the assessment of availability of drugs and equipment | The assessment of drugs, equipment and supplies could be simplified for example by focusing on fewer ‘tracer’ essential drugs and obtaining information on stock outs or unavailability rather than amount and quality of drugs. |
Link the assessment tool to existing quality of care frameworks and indicators | It needs to be made clear how the tool and the components assessed map onto the new quality of care framework and indicators being developed by WHO as well as other maternal and newborn health indicators being developed to measure quality of care at facility level. |
Consider alternatives to external assessors | Given the potential for assessor bias and for the presence of an external assessor to produce changes in provider behaviour (the Hawthorn or observer effect), the tool may produce more valid data if completed by staff working at the facilities. |
Revise the action planning process | An important drawback for practical use of the assessment data is that facility action plans were not completed It may be that this exercise is best coordinated at district level, where common problems with quality of care can be identified and resources targeted towards supporting facilities to make improvements. Staff at individual facilities may become overwhelmed if they try to tackle all the areas of care identified as in need of improvement. The colour coded dashboard (Fig. 1) may be a useful template for displaying data and could be used as a basis for discussion and prioritisation of problems for immediate and longer term action. |