Simulated patient encounters to improve adolescent retention in HIV care in Kenya: study protocol of a stepped-wedge randomized controlled trial

The Simulated Patient Encounters to promote Early Detection and engagement in care for adolescents (SPEED) trial aims to evaluate the effectiveness of the SP training intervention using a stepped-wedge RCT design in 24 HIV care clinics in Kenya. This pragmatic study design was chosen because this training intervention occurs at the clinic, rather than at the individual level, uses routine clinic data for primary outcome measurement, and would not be feasible to implement simultaneously at all facilities [31]. In this modified, one-way, cross-over design, all facilities eventually receive the intervention, which was important for government partners. Facilities are randomized to when the intervention is introduced, which allows for benefits of randomization [32]. All facilities will contribute intervention and control periods. The RCT consists of four intervention waves approximately 9 months apart. Each wave includes HCW participants from six facilities (Fig. 1). Once facilities receive the training intervention, they are considered “exposed” until the end of the trial.

We engaged NASCOP and county leadership during the planning phase of this study to identify eligible health facilities. We also established a study Community Advisory Board (CAB), including AYA leaders from networks of people living with HIV, to provide guidance on recruitment and retention of facilities and participants. The sampling frame included secondary and tertiary HIV care facilities from four counties in Kenya that were purposefully selected to represent different HIV epidemic patterns and services to enhance generalizability by geographic region and HIV prevalence. Eligible clinics had at least 40 AYA enrolled in care, a current electronic medical record (EMR) system, and no other special adolescent interventions. Facilities were excluded if anything would prevent the complete conduct of the training intervention at that site and/or the collection of outcome measures (e.g., discontinuation of the EMR system). Among 30 eligible facilities, we selected 24 facilities ensuring a balanced distribution of counties and facility sizes in each wave.

Electronic medical records from all AYA patients ages 10–24 years enrolled in HIV care at participating sites are eligible for data abstraction. A sample of AYA presenting for HIV care at enrolled sites are further eligible to complete patient satisfaction surveys. Eligible HCW participants are 18 years of age or older, employed at trial sites, and provide clinical and/or counseling services to adolescents. Individuals are excluded if they experience any conditions that would place them at increased risk of harm or preclude the individual’s completion of the study.

Facilities were randomized to one of four intervention waves (Table 1), stratified by county: Nairobi, Kiambu, Kisumu, Homa Bay and by facility size. County was used for stratification because it is a reasonable marker of differences in AYA HIV prevalence and HIV care services across regions [33]. All possible combinations that yielded an allocation of two Nairobi, two Kiambu, one Homa Bay, and one Kisumu site, half high (more than 73 AYA) and half medium volume (less than 73 AYA, based on the median number of AYA enrolled in all sites) in each wave were generated in Excel. One of the possible combinations was randomly chosen using a random-number generator for the randomization allocation.

Because this is a clinical training intervention using a pragmatic trial design, it is impossible to blind all members of the study team or trial participants to the randomization assignments. However, we have implemented procedures to minimize the number of individuals who are unblinded. Only the biostatistician has access to the randomization assignments of the 24 facilities. One month before each wave, the biostatistician provides a list of the six facilities in that wave to the study lead and the coordinator. All other facilities are informed that training will occur at a later wave. An independent External Advisory Panel reviews recruitment, enrollment, and potential for social harms annually during the trial. Because of the stepped-wedge design, there are no interim analyses or stopping rules.

We contracted with a casting agency in Kenya to recruit professional actors to be trained as SPs, hiring 14 Kenyan actors over age 18 years who could believably role-play AYA patients between 14 and 21 years. Before piloting the intervention, actors received a 5-day training facilitated by a consultant expert in the SP methodology. The training included review of seven case scripts, individualized coaching on role playing cases believably without over dramatizing, video-taped practice of the simulated patient encounters with actual HCWs followed by group discussion to refine the SPs’ portrayal of the cases, and training on providing verbal feedback to HCWs and completing the SP actor feedback forms. At the end of the training, the consultant and study team determined the final case assignments (Additional file 1).

Clinic leadership personnel are consulted to provide a list of up to 10 HCWs who provide adolescent services and to negotiate release from work to participate in the training and survey data collection. A study interviewer then approaches those HCWs to invite them to learn more about the study. HCWs provide written informed consent. HCWs are tracked over time to assess provider retention at clinical intervention sites, using a facility survey to monitor staff turnover. All AYA seeking HIV care on a given day are offered referral by a clinical staff member to our study team for survey participation. We schedule facility visits to “adolescent clinic” days or during school breaks to ensure adequate volumes of eligible patients. Oral consent or assent is obtained from AYA based on their status as emancipated minors. AYA aged 18 years and older, and those aged 14–17 years who are emancipated minors, provide oral consent to participate in surveys. Otherwise, parents/caregivers provide oral informed consent for AYA ages 10–17 years, and those adolescents provide oral assent.

The SP intervention is based on an adaptation of Kolb’s model of Experiential Learning [34], Bandura’s model of Social Learning Theory [35], and an adaptation of Andersen’s Behavioral Model of Health Services Use [36]. The 2-day training intervention focuses on skill-building through practice, with a series of SP encounters, individual feedback by SPs, and group debriefings. HCW participants receive orientation to the SP methodology, instructions for taking part in the encounters, and short didactic sessions on adolescent-friendly services in Kenya and motivational interviewing techniques [37]. After the didactic session, each participant rotates through seven SP encounters. Encounters are video-recorded and occur at a training venue with stations set up to resemble counseling rooms, separated by a wall or curtain for privacy. At the end of each encounter, the SP provides non-technical, emotional feedback. Facilitated group debriefings consist of review of videos and discussion about strengths and areas for improvement in each encounter.

Each case script follows a standard format, with a chief complaint, relevant medical and social history, and actor prompts (Additional file 2). Qualitative analysis of in-depth interviews with HIV-positive AYA enrolled in a study on HIV-testing services in Nairobi, Kenya informed the range of cases and chief complaints [38]. We also consulted World Health Organization (WHO) and Kenyan guidelines on “adolescent-friendly” services to ensure that the cases covered major AYA health concerns, including contraception and fertility desire (whether the client wants children in the future) [5, 13], sexual identity [39], mental health [40], HIV-stigma and disclosure [14], and gender-based violence [41]. These issues also are associated with poor retention in health services [5, 13, 42, 43]. Case scripts were reviewed by Kenyan general practitioners, pediatricians, and a psychiatrist with experience in AYA care; the training consultant; and SP actors for accuracy and relevance to local context. There are seven cases in total, including both male and female roles, and one case with an older sibling present as the caregiver (Table 2).

All training materials and data collection tools were pilot tested. The SP training intervention is implemented at the 24 facilities over four waves (Fig. 2, Additional file 3: Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) Checklist). Each intervention wave trains HCWs from six clinics, approximately every 9 months. We anticipate that training all HCW participants in each wave will take up to 4 weeks. Each wave includes HCW participants from all four counties. We repeat the training with groups of participants from each facility or county until all participants are trained; timeframes vary based on HCW availability and travel. The intervention training schedule is planned to minimize disruption to clinic operations and reflect a “real-world” in-service training.

Intervention fidelity is monitored throughout the trial. The SP training included practice-simulated encounters with actual HCWs, which helped to increase the realism and relevance of the case and identify areas where the actor needed additional training. We review a random sample of video-taped encounters in each intervention wave and rate actor fidelity according to a standardized checklist that has been adapted for this study [44]. We conduct refresher trainings before each intervention wave, and follow-up refresher trainings for actors who need additional support. We use standard operating procedures and standard training materials for the didactic sessions, track HCW retention in the training intervention, and track retention of SPEED-trained HCWs at study facilities over time.

The primary outcome for this study is AYA retention in HIV care at the individual level, defined as the probability of returning for first follow-up visit within 3 months of enrollment or returning for the first follow-up visit within 3 months after a “re-engagement visit,” among AYA who had been previously lost to follow-up (LTFU) (Table 3). A “re-engagement in care” visit is defined as a clinic visit after at least a 3-month absence from care. Because AYA who come for a re-engagement visit are effectively treated as new clients, this group is likely similar to newly enrolled AYA clients, and their inclusion will increase study power. Because there is no universal definition of LTFU [45] and the patterns of return visits are not well documented in AYA [4, 46], we will test an alternative definition of our primary outcome in a sensitivity analysis as return for any follow-up visit within 3 months among AYA currently enrolled in HIV care (i.e., patients “actively enrolled”).

Process and exploratory outcomes for the trial are described in Table 3. Process measures are collected using surveys to understand mechanisms of intervention effectiveness: HCW satisfaction, HCW competency, and AYA patient satisfaction with care. Exploratory outcomes to evaluate additional intervention effects on AYA outcomes are abstracted from EMR data. They include: antiretroviral therapy (ART) adherence (≥95% based on number of missed doses in past month by self-report. Refills within 1 week of scheduled return visit.); virologic suppression (<1000 copies/ml); linkage to Adolescent Package of Care services (e.g., family planning); number of hospitalizations; number of opportunistic infections; AIDS-defining illness; and mortality.

Outcomes for the health economic analysis are: cost-effectiveness, defined as the cost per additional HIV-positive AYA retained in care and, cost-utility of the intervention, defined as cost per disability-adjusted life-year (DALY) averted and cost per life-year saved. Additional exploratory outcomes for the cost-effectiveness analysis will include: cost per additional HIV-positive AYA with good adherence as defined in the EMR (≥95% or one or fewer missed dose in past month/dosing period. Refills within 1 week of scheduled return visit); cost per additional HIV-positive AYA virally suppressed; and cost per additional HIV-positive AYA linked to adolescent package of services defined by the Kenyan Ministry of Health (e.g., referrals for mental health counseling, family planning, gender-based violence care). Cost data will be estimated from AYA surveys, program records, and published literature, and include direct medical costs, direct non-medical costs, and indirect costs.

Eligible de-identified AYA clinic records from the EMR system are abstracted from each facility at baseline, and at the end of each wave to assess primary and exploratory outcomes (Table 3). Data for the primary outcome will be limited to records from newly enrolled and recently re-engaged AYA clients. The time period for data abstraction is 15 months before baseline up until 15 months after the wave 1-4 training intervention. Each data abstraction period covers 15 months to allow sufficient time to observe AYA retention outcomes or approximately two school breaks (about 3–6 months apart) when AYA typically come for HIV care. Data are abstracted and transferred to an encrypted external hard drive. Survey data are collected on electronic tablets using the Open Data Kit (ODK) software (https://​opendatakit.​org/​) [47]. The software provides an intuitive interface for validated data entry, automated export procedures for data downloads, and procedures for importing external data. Survey data are automatically uploaded to the secure study server when the device is connected to the Internet. Video-recordings are made of each HCW-SP interaction for debriefing purposes. Videos are reviewed and scored by a team of Kenyan and US-based experts to provide technical feedback on clinical competencies. Semi-structured exit interviews are conducted with a purposeful sample of up to 30 HCWs approximately 9–24 months after the SPEED intervention training to assess durability of new skills over time.

Sample size and power calculations used methods described by Hussey and Hughes [32]. Assuming 75% of AYA in the control period will return to clinic after first visit [6], a 15% dropout without replacement, a coefficient of variation (CV) of 0.25, and a two-tailed test with α = 0.05, 24 clusters and five time points, we would have 80% power to detect a 15% difference, or achievement of 90% retained in care, between control and intervention periods. The lower bound of the sample size required is estimated to be 720 clinic records (6 records/site × 24 sites × 5 time points). In the optimal scenario, with an average of 40 newly enrolled AYA clients at each facility per wave, and five data collection points, our estimated sample size would be 4800 clinic records. Under the same assumptions as above, we would have > 99% power to detect a 15% difference in retention between the intervention and control periods.

The primary analysis is intent-to-treat (ITT), assuming that AYA who receive care at clinics after the SPEED training will be “exposed” to HCWs trained in the SPEED intervention until the end of the study. No replacement of facilities will be used once the trial has begun. Baseline values of primary and process outcomes, facility characteristics, and socio-demographic characteristics of AYA and HCWs will be presented in descriptive tables.

We will use generalized linear mixed models (GLMM) to compare the probability of AYA retention in HIV care between the exposed (i.e., intervention) and unexposed (i.e., control) periods [48]. To estimate the effect of the intervention (Xij) on the individual level, we will use a GLMM model with a binomial distribution and log link, allowing for random effects for clusters (u _i ) and fixed effects for time (j). This approach models individual-level outcomes, adjusts for temporal trends, and accounts for correlation of outcomes within multiple levels of clustering (clinic, individual). We will estimate adjusted risk ratios (ARR) and 95% confidence intervals at an α = 0.05 level (two-sided). We will estimate the effect of the intervention on each secondary outcome using GLMM to generate ARRs. All models will be adjusted for clustering and time (c). The equation for a generic model is:

where i indexes the cluster (facility), j indexes time, and k indexes the individual.

We will conduct exploratory analyses to evaluate intervention mechanisms, non-constant temporal trends, and lag effects. We will examine whether an observed intervention effect may be explained through HCW post-training competency and AYA patient satisfaction scores. In that analysis, mean AYA satisfaction scores will be evaluated as predictors of AYA retention in care from EMR data. Temporal trends that change over time can confound the effect estimate in a stepped-wedge trial design [32, 48]. For example, new AYA HIV treatment guidelines may be introduced nationwide during the intervention that affect AYA retention. Time-varying temporal trends will be evaluated using methods recommended for stepped-wedge trials [48, 49]. The effect of the intervention may not be immediate or it may diminish. Intervention lag will be evaluated by including an interaction term between the intervention status (fixed effect) and time [48], and, separately, by using a fractional term for the coefficient of primary-effect estimate, which will reduce in magnitude at each time step [49].

Interview data will be analyzed using a constant comparative approach by two researchers [50]. They will code transcripts separately and compare results, resolving differences in application of codes through discussion. A codebook will be developed based on preliminary reading of the transcripts, and revised based on this iterative process. Patterns identified in the coding within and across interviews will be used to identify major themes around HCW uptake and application of skills learned in the training intervention. Results will be triangulated with the quantitative results in the study.

We will estimate the cost-effectiveness of the SP intervention in terms of cost per additional HIV-positive AYA identified and retained in care. The primary effectiveness estimate from the trial will be used for the number of additional AYA retained in care as a result of the intervention. Direct medical costs include laboratory tests, medication costs, other medical procedure costs, and personnel salaries. Direct non-medical costs include AYA transportation, food, and childcare costs. Indirect costs include opportunity costs from lost wages in AYA paid and unpaid work. We will also model the cost-utility of the intervention in terms of cost per life-year saved and DALY averted by maintaining HIV-positive AYA in care, including subsequent decreases in HIV transmission/acquisition. This will require linking effectiveness data to a mathematical model that estimates DALYs in AYA HIV progression, using a model adapted for AYA populations.

We will examine how these estimates vary through one-way and probabilistic sensitivity analyses in settings with varying AYA HIV prevalence, HCW wages, patient volume, and linkage to care rates. This will assist in identifying scenarios where the intervention is most cost-effective, allowing policy-makers to determine incremental costs and net benefits of the intervention. We will perform this analysis both from a limited societal perspective and from a Ministry of Health (healthcare sector) perspective. All benefits and costs will be discounted at 3% per year [51].