Assessment of training and mentoring for DR-TB care decentralization in Tanzania

We conducted a mixed study design using Kirkpatrick’s evaluation model (Fig. 1) [16]. The quantitative component of the study consisted of: characteristics of DR-TB treatment initiation sites and trained HCWs, pre- and post-test scores, end-of-training evaluation results, and DR-TB treatment outcomes as a proxy for HCWs performance. The qualitative component consisted of: the DR-TB teams’ self-assessment, focus group discussions with the team, and the supervisor’s interview. The quantitative component was carried out first to assess the quality of training followed by qualitative interviews to elicit health workers perception, attitudes and practices on provision of decentralized DR-TB care.

The DR-TB treatment outcomes were assigned by a team of physicians working at Kibong’oto Infectious Diseases Hospital based on the patient’s progress (i.e., based on adherence to treatment and signs of clinical improvement), and mycobacterial culture results. The treatment outcomes were recorded as cured, treatment completed, died, treatment failure, lost to follow-up or not evaluated as adapted from WHO DR-TB definitions [17]. Favorable treatment outcome: was defined as a combination of treatment completed and cured and unfavorable treatment outcome means a combination of deaths, lost to follow, culture positive and not evaluated. Where available, culture was used classify the interim (culture conversion at month six of treatment) and final treatment outcomes at the end of treatment.. During the time of the study, the cultures were performed at the Central TB Reference Laboratory (CTRL) and at zonal TB laboratories using Lowenstein–Jensen (LJ) medium with a turnaround time of 3–8 weeks as per Tanzanian operational guidelines for management of drug resistance TB [18].

The interviews for HCWs were conducted at the one national central site (Kibong’oto Infectious Diseases Hospital) and 14 decentralized sites which were selected purposefully to include sites with high patient volumes, treating 286/327 (87%) of patients in 2016 and 2017 when the study was conducted. Treatment outcomes of 327 DR-TB patients were analyzed for 2016 and 2017 to explore how training supported the level of care in the decentralized sites.

This study included 454 HCWs from 93 DR-TB initiation sites that were trained between January 2016 and December 2017 by a team of 23 national trainers. Outcomes of patients enrolled in the year prior to training (2015) were used for comparison. The performance of 55 sites out of 93 sites was assessed as these were providing DR-TB care by December 2017. DR-TB teams from 15 sites were interviewed in a focus group discussion, 14 of which completed the self-assessment questionnaire and 14 had their supervisors interviewed to collect qualitative information about DR-TB teams’ performance on the job (Fig. 2). Interviews were conducted in 2018 and data analysis was done in 2019. This long duration after the training was to accommodate for outcome results of trained sites that take 24 months for the whole cohort to complete treatment after treatment initiation.

As part of the decentralization process, the NTLP selected the decentralized sites, assessed their readiness to start DR-TB care, supported improvement of the sites’ infrastructure and equipment, and built HCWs capacity to provide DR-TB diagnosis and care through a didactic training and on-job mentoring (Fig. 3).

The didactic training consisted of a 5-day comprehensive training using the newly developed “training modules for DR-TB initiation sites” (Additional file 1). The training methodology was competency based and interactive. We trained an interdisciplinary DR-TB team consisting of: a Regional TB and Leprosy Coordinator (RTLC), a District TB and Leprosy Coordinator (DTLC), a Clinician, a DOT nurse, a Laboratory Technician/Technologist, a Pharmacist, and a Social Welfare Officer.

On-job mentorship started once a DR-TB patient is diagnosed in their respective district, a mentoring team from KIDH, provided an additional 5-day mentoring on bedside clinical and nursing care and second-line drugs supply chain management.

Treatment of the DR-TB patients was done according to Operational Guidelines for Management of Drug-resistant TB in Tanzania, second Edition (2015) [19].

Quarterly supervision was provided by a team of four people from national, regional and district-level staff. During the supervision, the identified gaps were mentored on site. The next quarter supervision team also made a follow-up of previous visit action points before starting a new supervision.

For qualitative data collection, we used semi-structured interviews for DR-TB team self-assessment and supervisors and a tool for DR-TB team focus group discussions interview. Qualitative data were collected on HCWs performance and the relevance of the training and mentoring one year after the training was conducted to the 15 DR-TB sites. All available members of the DR-TB teams in 15 sites participated in the focus group discussion interviews at the DR-TB health facility. The interviewer (BS) was an independent researcher not involved in DR-TB care, training or mentoring. BS interviewed and facilitated focus group discussions, making use of standardized tools (Additional file 1). KNCV officer (DL) collected and analyzed the questionnaires filled by the trainers of trainers (Additional file 1). DR-TB team members of 14 sites self-assessed their performance and 14 of these DR-TB teams had their supervisor participate in the semi-structured interviews. Collected transcripts from semi-structured interviews were coded by two authors MH and DL. Coded inputs were then categorized to generate themes using thematic analysis.

For the quantitative component, characteristics of the DR-TB treatment initiation sites, enrolled DR-TB patients and trained HCWs were collected. Training impact was assessed using the Kirkpatrick’s evaluation model. Interim and final DR-TB treatment outcomes were compared between the central and decentralized sites and were extracted from the DR-TB enhanced cohort review system. Frequencies per demographic variables such as cadre and gender were presented for the different training groups. Training evaluation scores for method and content modules were measured on a Likert scale: 1 = poor, 2 = satisfactory, 3 = very good and 4 = excellent. The pre- and post-training test scores were compared before and after the training using comparison of means and a paired t-test. A p-value of < 0.05 was considered statistically significant. Multivariable logistic regression was used to determine if training was a predictor of outcomes. Data analysis was performed using Stata Software version 16.1 (StataCorp LP, College Station, TX, USA). We used a composite variable for the final outcome whereby favorable outcomes were a combination of cure and treatment completion rates and unfavorable outcomes comprised death, lost to follow-up, not evaluated and treatment failure (Fig. 4).

The number of DR-TB initiation sites increased from 1 national DR-TB site (KIDH) to 55 sites from January 2016 to December 2017. Decentralized DR-TB sites are distributed all over the country, with a concentration in the Eastern and Northern part of the country. Most sites are in urban areas where most patients are found (Additional file 1). In our study, we selected 15 sites to evaluate the effect of training and mentoring on HCWs performance (Additional file 1). These sites treated 286/328 (87%) DR-TB patients in 2016 and 2017.

A total of 449 MDR-TB patients were enrolled in treatment between 2015 and 2017. Decentralization begun in 2016 with 23% of patients in decentralized sites which increased to 62% in 2017. During the same period, 430 trained HCWs were analyzed for performance of the training and mentoring after excluding 24 HCWs who missed their results (Fig. 2, Table 2).

The distribution of the patient characteristics between patients treated in the central and decentralized sites were comparable for age, sex, treatment history, HIV status, and nutritional status, but not for patient residence as the patients treated at the central site were more often from a rural residence in 2017 (Table 1).

Out of these 430 trained HCWs, 226 (53%) were male, 132 (32%) were regional and district TB program coordinators, 33 (8%) TB direct observed treatment (DOT) nurses, 43 (10%) laboratory staff, 73 (17%) other clinicians, 78 (18%) other nurses, 39 (9%) pharmacists, and 28 (7%) social workers (Table 2).

A multidisciplinary team of 21 national trainers was formed: clinicians (10), nurses (7), and laboratory staff (4). Eleven out of 21 trainers, participated in this study and were; clinicians (6), laboratory staff (3) and nurses (2).

The pre-training/post-training test scores showed that trained HCWs had an average increase of 15–20 points during the course and this was statistically significant with p < 0.001 (Table 2).

Participants’ end-of-training evaluation results showed that most trained HCWs highly appreciated the course content and methodology (Fig. 5). The modules “Health Education”, “Supportive Supervision” and “Recording & Reporting” had highest scores, however the score differences among the modules were limited. The score differences among the training groups were not substantial.

449 DR-TB patients initiated treatment from 2015 to 2017. For interim results (after 2 months of treatment) decentralized sites had high proportion of culture conversion (72%) compared to centralized site (50%) in 2016, however there was no difference in the 2 month culture conversion between central and decentralized sites in 2017. After 6 months of treatment, culture conversion was comparable between decentralized and centralized sites in 2016 and 2017. For final outcomes, cure rates were higher for decentralized (86%) than KIDH (58–69%) in 2016 and 2017. Likewise, deaths and loss to follow-up were lower for decentralized sites compared to KIDH in the same years (Table 3).

Comparing predictors of unfavorable outcomes in decentralized vs centralized site in 2016 and 2017 (Table 4), we found BMI (< 18.5 kg/m²) was a significant predictor of unfavorable outcomes among patients at the national central site [adjusted OR–aOR; 3.1, p = 0.005] similarly to HIV-coinfection among decentralized treated patients (aOR; 4.2, p = 0.02). Older ages [31–50 years (aOR 2.2, p = 0.02) & > 50 year (aOR 2.6, p = 0.02) and centralized treatment delivery (aOR 2.9 p = 0.006) were associated with unfavorable outcomes.

The trained HCWs of the 15 DR-TB sites qualified the training course as comprehensive, relevant, good, practical and helpful and that it did not focus on clinical topics only. The course gave them confidence and skills to provide treatment to DR-TB patients. Five DR-TB teams mentioned that the “New Drugs and Regimens” session was confusing. Teams suggested regular updates on (the frequent) changes in DR-TB care and train new staff members which are needed due to frequent staff rotation. (Additional file 1).

All 11 trainers evaluated the quality of the training as good: as it included all aspects of decentralized DR-TB care and perceived that trained HCWs have learned from this course. Eight out of 11 trainers assumed that the trained HCWs were ready to initiate DR-TB treatment. Three out of 11 trainers expressed their hesitations mainly due to lack of adequate facility equipment and infrastructure and insufficient Infection Prevention and Control practice. Some trainers observed that trained HCWs lacked confidence in the quality of care at their facility and fear DR-TB because they have no experience in treating DR-TB patients and assume having a high risk to be infected by them (Additional file 1).

Supervisors assessed DR-TB teams’ performance after the training as very good (5 supervisors) to good (10 supervisors) concluding that trained HCWs had developed their DR-TB knowledge during the training (Additional file 1).

Nine of the 15 DR-TB sites received mentoring after DR-TB training, of which four teams felt they were inadequately mentored, and two teams perceived the mentoring more as control than support of the DR-TB team. During mentoring with facility multidisciplinary teams, the interviewer noted the frequency and duration of the mentoring varied from team to team between a ½ h visit to more than 5 days on the job training and, from when a new patient comes to four times a year. There was no standardized mentoring approach and the mentoring was strongly focused on the clinicians and DOT nurses. Mentoring and supportive supervision were perceived the most appropriate way for most DR-TB teams to develop further their competencies. DR-TB teams mentioned They showed how to initiate treatment and brought common understanding among the team and We saw how the mentor performed in handling patients and medication; he built our confidence.

The four trainers that had also a mentoring role, mentioned that mentoring has reinforced what the trained HCWs learned in the training because their workplaces afforded them with more comfortable learning environment, was more hands-on and focused on observed HCWs gaps.

The DR-TB teams self-evaluated their performance between fair and excellent on the different aspects of quality performance (Additional file 1). There were no substantial differences among the DR-TB teams. Teams were most satisfied with their performance on (1) confidence to provide DR-TB care; (2) ensure availability and quality of DR-TB drugs; (3) give health education and (4) provide patients support. Teams had concerns about their performance on (1) timely and accurate laboratory test results; (2) timely and quality clinical care; (3) quality recording and reporting and (4) quality and regular supportive supervision. The teams proposed to train more HCWs, include facility managers in the training and provide ongoing mentorship. They suggested to update HCWs regularly on new developments, increase punctuality to provide laboratory results and involve the whole laboratory team in this. They proposed monthly DR-TB meetings, training and mentoring to ensure that all HCWs are competent and up to date.

DR-TB teams’ supervisors frequently mentioned the DR-TB teams’ strengths in treatment initiation, providing the correct drugs and drugs dosage, patient management and care, laboratory investigations and patient follow-up. The main challenges that DR-TB teams faced according to their supervisors are mainly in the field of stigma, facility infrastructure, drug delivery, delays in culture results, HCWs shortage and HCWs capacity building (Table 5). Supervisors suggested to further build DR-TB team capacity in patient data entry, management of side effects and short regimens. DR-TB teams need to develop their competencies further by practice to gain confidence. The team doesn’t get many patients and needs more practice in initiating treatment, said one of the supervisors. Regular and quality supervision, reminding HCWs of what they have learned in the course, is a must for professional development. Continuous medical education for HCWs that were not trained yet and sharing of documents were also brought forward.