Classification and rates of adverse events in a Malawi male circumcision program: impact of quality improvement training

Utilizing a clinical case conference approach, a group problem-solving, QI project involving retrospective chart audits, case-conference AE classification, and provider training was conducted at a VMMC clinic in Lilongwe, Malawi. Evaluation of the QI project involved a quasi-experimental pre/post design.

Bwaila VMMC Clinic is operated jointly by the University of Washington International Training and Education Center for Health (I-TECH) and the Malawi Ministry of Health (MOH). The US Centers for Disease Control and Prevention (CDC) in Malawi provides technical and programmatic support. The clinic opened in September 2012 and performs approximately 1000 VMMC procedures per month. The clinic is located at a district hospital near the city center in Malawi, where nurses and clinical officers are trained to provide VMMC services using the forceps-guided technique [21].

Of 894 clients undergoing VMMC in November 2012, 721 (81 %) returned for their scheduled 48-hour post-operative visits, 491 (55 %) returned for their 7-day visits, and 207 (23 %) returned for their 6-week visits. This was representative of client follow-up throughout the study period.

A multi-disciplinary team of seven clinicians, monitoring and evaluation staff, technical advisors, and program managers audited the first 3000 client records for initial review (the amount feasible in a 2-week audit). Charts consisted of standardized forms provided by the MOH. Information collected during the audit included client’s reported reason for seeking VMMC, clinician-surgeon name, duration of operation, and any indication of a possible AE. In charts where possible AEs were identified, the team further recorded any available clinical information including: detailed clinical notes, timing, severity, progression, treatment, and outcomes.

Each day during the 2-week period, a case conference was held with the entire VMMC program clinical team and the multi-disciplinary audit team. Consistent with previous clinical trial and the President’s Emergency Plan for AIDS Relief (PEPFAR) definitions, the team jointly classified each event as not related, possibly related, or definitely related to the VMMC procedure. Each possible or definitely related AE was further classified as: mild (requiring only reassurance, monitoring, or local wound care); moderate (requiring clinical support such as antibiotic medication or additional suturing); or severe (requiring return to the operating theater or hospitalization). Each case was also reviewed to assess whether any treatment provided was appropriate, and to determine lessons learned for future management. These methods are generally consistent with recommendations from the MOH and approaches used in two of the three randomized clinical trials of VMMC to prevent HIV infection [1, 2, 21].

During data collection and case conferences, providers were also queried regarding lessons learned, challenges in provision of care, and client flow was traced through the clinic. After baseline evaluation, clinicians and managers developed a QI plan to improve prevention, assessment, and management of AEs. A repeat audit was conducted 6 months later, and proportions and severity of AEs before and after the QI intervention were compared.

Data were analyzed using STATA SE v11.0 (StataCorp, College Station, TX). Summary statistics were calculated, and comparisons of AEs before and after the intervention utilized chi-squared tests of proportions.

Clients consented to VMMC services through written informed consent, and evaluation data were collected using routine program tools. The Malawi National Health Sciences Research Committee provided a letter of approval for program evaluation and dissemination, and the University of Washington Human Subjects Division determined that study activities did not meet the regulatory definition of research under 45 CFR 46.102(d).

During baseline audits, we reviewed 3000 charts. Although nurses and clinical officers are both trained to circumcise clients in Malawi, the clinical officers conducted >90 % of all surgeries. Client reason for presentation was only available for the first 2192 clients due to changes in MOH forms. Overall, 67 % reported seeking VMMC for HIV or sexually transmitted infection (STI) prevention. Hygiene (11 %), medical (5 %), personal (5 %), and religious (4 %) reasons were also cited. A small number indicated cancer prevention (Table 1).

Of 3000 procedures, we initially documented 418 (13.9 %) possible AEs. As we began recording cases, we found that most AEs were recorded as “infections” (5.4 %), with varying descriptions ranging from, “mild inflammation,” to, “pus at the frenulum.” Nearly all of these, as well as some not identified as AEs or infections in the chart, were treated with antimicrobial drugs. The choice and dose of antibiotics was surprising (e.g., oral ciprofloxacin and metronidazole, plus intramuscular gentamicin), until we learned that the staff had attended training on STI management and we located the national guidelines for urethral discharge, which included all of these drugs. We also inspected drawers containing available antimicrobials and found that the only antimicrobials available at the clinic were those listed in the STI formulary.

We used sequential client identification (ID) numbers to explore changes in rates over time. Spikes in rates of AEs were consistent across VMMC providers, however many cases appeared similar to the audit team. Although the spikes were not associated with a particular surgeon, they appeared to be related to the same clinician at the post-operative visit. While MOH forms did not have a space to enter post-operative visit clinician name or initials, the handwriting was easily recognizable to the audit team. We established a ‘case definition’ to explore these spikes as if it were an ‘outbreak investigation,’ where cases met all of the following conditions: same clinician handwriting, ‘infection’ diagnosed at post-operative day 7, treated with ciprofloxacin and metronidazole, and then ‘resolved’ at the next visit. We re-reviewed all ‘possible AE’ charts and found that 289 (69 %) of the 418 met this definition. Although the program employs more than 20 nurses and clinical officers, 152 (52.6 %) of these cases were from the same nurse and highly suspicious for misdiagnosed infection. Further evidence included two charts where a second nurse intervened and documented withholding antibiotics. During the audit process, clinicians would also ask the consulting surgeon to review patients identified as having possible infections. The surgeon found that none of these clients showed signs of infection, rather were displaying mild inflammation consistent with the healing process. We therefore excluded the 152 cases as misclassification of infection.

Of the remaining 266 AEs, the team concluded that 257 (97 %) were procedure-related. Among the 257 procedure-related AEs (8.6 per 100 VMMC procedures), 6 (0.2 %) were classified as mild, 218 (7.3 %) were moderate, and 33 (1.1 %) were severe. Although some of the cases may have been mild in presentation, the severity classification system in use by VMMC programs assigns a severity level based on the intervention and not necessarily the presentation. Therefore even if the team assumed a mild presentation based on the clinical report, a patient treated with antimicrobials was, by definition, assigned a moderate AE.

Case conference reviews concluded that 89 % of AEs were not treated appropriately; the majority of these were prescribed antibiotics for mild symptoms. Among 45 cases deemed not to be AEs during case review, 36 (80 %) had been prescribed antibiotics; 19 of 24 cases (79 %) designated as appropriately treated were given no antibiotics, and all 170/170 cases (100 %) inappropriately treated were given antibiotics (median two drugs per AE case).

Several structural factors were found to contribute to the rates and misclassification of cases. Provider prescription practices for management of post-operative inflammation were consistent with national guidelines for urethral discharge, available antibiotics were the STI formulary, and providers reported feeling well-trained in surgical care but insecure in skills related to post-operative assessment and care. Wound care supplies, such as normal saline, irrigation kits, or gauze, were limited in availability or stored in another room, while antibiotics were in a nearby drawer.

Although not the primary assessment of this audit, a number of data-quality issues were identified. Client identification numbers were assigned as clients were entered into the national VMMC registers in the recovery room. The client ID number corresponded to the row number in the register, however in several cases, the clinician had incorrectly entered or reversed the digits of the ID number when moving to the next page (e.g., 2441 becomes 2401), resulting in a large number of clients with two different ID numbers, or different clients with the same ID number, and a program miscount of the number of procedures. Other common issues included: missing data as chart notes described signs and symptoms of AEs without checking the box indicating an AE was identified, database overwrite issues as duplicate ID numbers were entered, and version control with old versions of forms being printed as supply of current forms ran out.

Clinical and program teams implemented a QI plan based on several identified challenges. The first strategy was grouped around post-operative assessment and management. Providers expressed a lack of understanding of what a normal healing adult circumcision should look like and a need for a refresher in basic wound care training. Also of concern was the limited availability of basic wound care supplies. Staff expressed a desire to prioritize patient reassurance and basic wound hygiene before giving antibiotics, and agreed to query a second clinician prior to prescribing antibiotics. Onsite technical in-service surgical training in circumcision technique, assessment of healing, and proper wound care was also provided. The second strategy involved improvements in data entry and classification of AEs. Providers agreed to continue the case conference process, reviewing each potential AE both for treatment and classification in order to enhance clinical practice.

A repeat process 6 months after implementing the QI program evaluated 2540 charts and identified 190 clients (7.5 %) with possible AEs (Table 1). Subsequent discussion in case conferences classified 123 clients to have had an AE, including 115 considered procedure-related events (4.5 per 100 VMMC procedures) (Table 2). Of these 115 procedure-related AEs: 67 (2.6 %) were classified as mild, 28 (1.1 %) moderate, and 20 (0.9 %) severe. Program-reportable (moderate and severe) AEs totaled 48 (1.9 per 100 VMMC procedures). Severity distribution and overall number of reportable AEs were significantly reduced (both p < 0.001). The follow up audit also identified changes in specific AE types, with fewer cases of infection (5.4 % before vs. 1.8 % after) and a higher proportion of bleeding cases (0.3 % before vs. 1.5 %) after the intervention. Overall rates of total AEs had decreased by 48 % (from 8.6 to 4.5 per 100 VMMC procedures, p < 0.001) and moderate-plus-severe (program-reportable) AEs decreased by 75 % (from 8.4 to 1.9 per 100 VMMC procedures, p < 0.001).