Effectiveness of educational outreach in infectious diseases management: a cluster randomized trial in Uganda

Sub Saharan Africa experiences 37 % of the total deaths attributable to infectious diseases globally [1], but contains only 3 % of the world’s health workers [2]. The shortage of physicians in sub Saharan Africa is at critical levels, with only 2 physicians per 10,000 population compared to 28 in high-income regions [3]. This shortage of physicians has led to many physician roles, such as initiation of antiretroviral therapy shifting to mid-level practitioners (MLP) with fewer years of medical training [4], formally [5‐7] and informally [8, 9]. Yet as reported in one Ugandan study, many of these MLP have not been trained to take on these tasks [10]. They therefore require capacity building to meet the demands of changing health policies and guidelines [11]. Some studies have shown that with additional training MLP can competently execute physicians’ roles and, in some instances, match their performance [12‐14]. However, most trainings for MLP are classroom-based, often taking them away from their workstations [15, 16], disrupting workflows and limiting patients’ access to care. Several systematic reviews show that classroom-based training may not equip the attendants with sufficient skills to ensure adherence to clinical guidelines [17‐19].

There is increasing interest from experts and donors in favor of educational outreach and continuous quality improvement (CQI) approaches [20‐24] that integrate across diseases [25‐28]. Facility-based educational outreach trainings increase opportunities for team-based interactions during MLP learning [20, 29] and reduces MLP time away from health facilities. Several studies have shown that educational outreach both with and without CQI activities can improve the quality of patient care [5, 30‐35]. It may also build capacity in patient care for almost all providers, without increasing workload on the MLP who do not attend trainings.

Between 2009 and 2010, the curriculum development and mobile teams of the Integrated Infectious Diseases Capacity Building Evaluation (IDCAP) developed and delivered two novel interventions to build capacity for treatment and prevention of infectious diseases: 1) the Integrated Management of Infectious Disease (IMID) course with distance learning aspects, taught at the Infectious Diseases Institute in Kampala, Uganda, and 2) facility-based on-site support (OSS) visits, an educational outreach intervention with CQI activities [29]. Implemented as a mixed design with pre/post and cluster randomized trial components, IDCAP tested the effect of IMID training and OSS on individual clinician competence [36], practice [37], mortality among children less than five years of age [20, 38], and facility performance [33, 39]. IMID was associated with a statistically significant 9.8 % absolute increase in clinical competence, whereas OSS was not associated with an incremental improvement in the same outcomes. [36]. For clinical practice, IMID was associated with statistically significant improvements in taking a patient’s history and physical examination, and OSS was associated with an incremental improvement in these tasks [37]. Child mortality increased in both arms, but the incremental effect of OSS delayed the increase during the intervention [38].

In the main trial analysis of facility performance, [33] Weaver et al. reported the effects of IDCAP’s IMID and OSS on 23 facility performance indicators. Facility performance refers to the process of care, and may reflect the contributions of more than one provider at a facility. For example, the proportion of malaria suspects with a malaria test result recorded (Indicator 4), reflects whether or not the clinician ordered the test, and the laboratory professional performed the test and recorded the result. Of the 23 indicators, 12 were based on a data surveillance system with data on each outpatient visit. The other 11 indicators were based on Ministry of Health Registers for HIV care, maternal, child and newborn health and tuberculosis. The 12 indicators based on the data surveillance system are presented in Table 1, and include three indicators for emergency triage assessment and treatment, four indicators for malaria case management, four indicators for case management of respiratory illness, and one indicator for HIV testing.

Among those 12 indicators, Weaver et al.’s reported that IMID alone was associated with statistically significant pre/post improvements in three indicators: outpatients triaged (Indicator 1), emergency and priority patients who were admitted, detained or referred (Indicator 2), and pneumonia suspects under age five assessed for pneumonia (Indicator 8). A combination of IMID and OSS was associated with statistically significant improvements in four indicators: outpatients triaged (Indicator 1), emergency and priority patients who were admitted, detained or referred (Indicator 2), 3) estimated malaria cases who received appropriate antimalarial treatment (Indicator 5), and patients with a negative malaria test result who were prescribed an antimalarial (Indicator 6), for which a decrease was an improvement. The incremental effect of OSS was not statistically significant for any of the 12 indicators, but many of the effect sizes were large.

Also, during the ethical review of the IDCAP protocol, the School of Medicine Ethics and Research Committee of Makerere University expressed concern that the interventions would affect the distribution of the workload. The data surveillance system data provided an opportunity to examine whether workload increased among the IMID-MLP with OSS, and among the No-IMID health workers with OSS. We therefore tested the effect of OSS on changes in workload in each subgroup of providers.

There have been limited rigorous randomized education trials that demonstrate the effects of health worker training approaches on patient care outcomes in the resource constrained world. In this study, we have evaluated the effect of an educational outreach intervention involving nine monthly on-site support (OSS) sessions on health facility performance in two groups of trainees – those who were provided with five weeks of off-site classroom education and those who did not have this opportunity.

The OSS intervention did not significantly increase patient load within the subgroup of health providers who did not participate in the IMID training. Among the mid-level practitioners (MLP) who received both IMID training and OSS, the number of patients per provider per calendar day increased; but the changes in workload attributable to OSS were not statistically significant. The effects of OSS on facility performance indicators were heterogeneous. Patients treated by health providers who attended the IMID training benefited from improved case management of malaria and respiratory infections. Patients treated by providers without the IMID training benefited from improvements in emergency triage, assessment and treatment.

The number of patients per provider per calendar day was much lower than we expected. On average, the health providers with IMID training saw 4.64 patients per calendar day, and the other providers 0.83 indicating that the MLP saw 5 times more patients than other providers. This was expected since MLP involved in daily management of patients in the outpatient clinics and spent over 80 % of their time at the health facility seeing patients were prioritized during the selection process [20]. These results for the MLP with IMID training are consistent with those of the Institute for Health Metrics and Evaluation’s report that showed health providers saw an average of three to five patients per day [43]. These results also indicate that health workers in Uganda treat very few patients per day, but this finding contrasts with other reports in which complaints of heavy workload have been cited among health workers in Uganda [44] and other sub-Saharan Africa countries [45, 46]. The reported number of patients seen per provider is an average across the estimated working days for each of the study time periods. It is possible that providers did not work on all estimated days and thus treated a higher number of patients on days when they worked [47]. In a review of studies in six countries including Uganda, it was found that 35 % of the health care providers were absent during their assigned shifts [48]. Among many factors, the absence usually arises because health workers are on leave, attending trainings off-site, tending to their gardens, or doing work in private clinics to supplement their income [47]. If health workers are only on duty an average of 65 % of the time, then the average number of patients per provider per day would increase by one-third. It is also possible that the number of patients’ visits per day was higher on some days than others. Adding in the fluctuations in patient visits and it possible that some clinicians would manage much higher number of patients on some days. Further analysis is required to determine how the number of patients per day seen by clinicians varies by day and across facilities.

Among the health workers without the IMID training, this analysis showed no statistically significant effect of OSS on changes in the volume of patients they managed. In fact, the number of patients seen per provider per calendar day among these health providers remained the same before and during the OSS. Yet a slight increase in volume of patients seen per provider per calendar day among the MLP with IMID training was observed during OSS, although this increase was not statistically significant. This increase in patient load among IMID trainees coupled with no increase among providers who did not attend IMID training, shows a general increase in patient volume overall during OSS, perhaps pointing to an appreciation of better quality services being offered at the health facilities. Rather than shift tasks and workload among cadres, OSS sought to improve facility performance within each cadre’s scope of practice. This could explain why there was no significant increase in patient volume among health workers without IMID training and decrease among health workers without the IMID training.

Overall, OSS was effective for three of the 12 facility performance indicators in this subgroup analysis, compared to none in the main IDCAP trial results [33]. In this subgroup analysis, only records with complete data on the provider’s cadre and IMID attendance were analyzed unlike in Weaver et al.’s [33] analysis that included all patient records. It is plausible that providers with incomplete data also attended fewer OSS or benefited less from them and diluted the effects. We saw evidence of the no-IMID subgroup diluting the OSS effects of the MLP with IMID especially for indicators 11 (patients with AFB smear negative results who received empiric treatment for acute respiratory infection) and 16 (patients with an HIV test result recorded). On the other hand, evidence of the MLP with IMID diluting the effects of the no-IMID subgroup was observed for indicator 2 (emergency and priority patients who were admitted, detained or referred).

The effects of OSS among the MLP with IMID tended to be statistically significant for indicators that reflected clinical decision-making, such as prescribing antimalarials and antibiotics. In contrast, the effects of OSS among the no-IMID subgroup were for emergency, triage and assessment indicators that required a broader team effort to identify and manage patients appropriately. Although OSS did not seek to change a cadre’s scope of practice, it did seek to empower all providers to triage patients and organize immediate care for emergency and priority patients.

The effects of OSS observed in this analysis were heterogeneous across indicators as well as across subgroups. In the overall sample, the average effect size was 34 % and higher than estimates elsewhere. For instance, reviews of continuous medical education and effect of educational outreach visits on health workers’ training reported median improvements ranging from 6.9 to 13.9 % [19] and from 5.6 to 21 % [49] respectively. The effect sizes for the emergency, triage and assessment and malaria case management indicators were generally larger than the effect sizes for case management of respiratory infections and HIV testing. The indicators for the first two coincide with the first two OSS. The OSS sessions for the latter two were delivered later in the intervention, an indication that perhaps, the expectation of immediate improvements over a shorter period of time may have been unrealistic. There is need for monitoring these indicators over more months to observe whether facility performance will improve over time.

Also future researchers should consider designing their trials for a subgroup analysis. Within resource-limited settings in many sub-Saharan African countries, clinicians and other higher level cadres are the ones usually selected to attend continuous medical education trainings to support them improve the quality of diagnosis and treatment. Allied health workers, nurses and other lower level providers often miss out on such opportunities since they mainly play supporting roles to clinicians in providing patient care. OSS would therefore be a great opportunity to improve capacity of these non-clinician health workers to dispense better quality patient care in the entire patient care process. For clinicians and higher level cadres, OSS would be important in supporting them to apply knowledge and skills in diagnosis and treatment acquired from classroom training.

To our knowledge, the IDCAP trial is one of the first randomized trials that evaluated the effects of an integrated educational outreach and quality improvement intervention in sub-Saharan Africa. The results of this study therefore provide important evidence that could inform capacity building interventions and policies for health workers in resource limited settings.

The MLP with IMID training managed thrice as many patients per day compared to other providers. Out of the 12 facility performance indicators, OSS improved three indicators across all patients and among patients managed by MLP with IMID training, but only two indicators among patients managed by providers without IMID training. These results show that OSS supported MLP who diagnosed and treated patients to apply knowledge from IMID. For other providers, OSS supported team work to manage emergency patients. Since provision of patient care is a team effort with clear roles for every health worker, health workers engaged in clinical diagnosis and treatment may need OSS in addition to classroom training, while the nurses and other providers who play a supporting role to the clinicians will need OSS to effectively contribute more to patient care.