Implementation of a stepped-wedge cluster randomized design in routine public health practice: design and application for a tuberculosis (TB) household contact study in a high burden area of Lima, Peru

This protocol is a joint endeavor between local NTP partners including the operational managers of the Ministry of Health TB program in SJL and researchers. Indeed such collaborations at the outset of operational studies, like the stepped-wedge CRCT in SJL, are crucial to their success. During the planning stages, a partnership of NTP programmers with local and international researchers was formed with the common goal of estimating the added benefit of actively evaluating HCs of TB cases for disease compared to the existing DOTS TB strategy of symptomatic individuals self-reporting for evaluation (standard of care). NTP programmers highlighted the importance of having a rapid intervention rollout in order to meet annual programmatic targets and to demonstrate public health action across all health centres in their jurisdiction. Typically in many RCT designs, only half of the included study health centers would be assigned to the intervention arm [8]. Researchers emphasized the need to plan study design aspects such as random assignment, sufficient study sample size in intervention or control arms and supervision of health workers to ensure high quality implementation and sound data management. The stepped-wedge design most aligned with stakeholder needs, including the introduction of randomized crossover times and allowing NTP programmers to oversee a controlled implementation of the intervention across all health centres [26]. This novel design is considered particularly useful in the implementation of public health interventions, given the phased implementation steps. However stepped-wedge designs are relatively new, and many programmatic managers may be unfamiliar with their ability to yield methodologically valid and informative results. Finally, an upfront understanding of the utility of this implementation design is needed so findings of this study would considered by local decision makers hearing of the stepped-wedge CRCT design for the first time.

Researchers can advise on methodological design issues in order to meet the needs of the research question, however in order to understand the operational research aspects of the protocol, gaining a thorough understanding of the local daily operations of the TB program at the health centre level is a key aspect to the design of this protocol. This knowledge included understanding of how contact investigations are undertaken routinely, how the intervention will be applied in local NTP centres, and how monitoring, supervision and data collection could be undertaken feasibly throughout the study period.

The stepped-wedge design is particularly useful in this study due to the required widescale roll-out of this public health intervention. In SJL, the systematic stepped-wedge implementation of a provider-initiated ACF program amongst HCs of TB cases provides new evidence, where previously very limited pre-existing effectiveness data are available and few, if any, have been measured within the local public health program in a TB endemic area.

While the staggered intervention initiation times increase the flexibility of the stepped-wedge design in practice, it also requires understanding of the methodological complexities involved. A stepped-wedge allows for a degree of flexibility in its design, such as the number of centres at initiation, and enough time between wedges to provide training and achieve enough study power. However, this design is still subject to biases and threats to sample size and study power. Determining the number of clusters, the number of steps and time-frame for rollout must all be predetermined and require special consideration. If the composition of clusters to be randomized at each crossover time point is unbalanced, this could result in skewed increases or decreases in the measured outcomes. For example, if all high TB rate centres were to cross over at the first time point and all low TB rate centres cross over at the last time point, this could result in a skewed numbers of secondary TB outcomes in HCs observed earlier and for a longer time period of the study. A simple stratification for the randomization of health centres by TB rate at each crossover time point is used in the current protocol to account for TB burden and as a proxy for the corresponding size of clinics to evenly distribute centres across the various wedges [30, 31].

A common challenge for many operational research designed studies is that blinding of the intervention assignment is not possible. In the current protocol, health centres awaiting crossover may have anticipated their likely crossover time based on the number of health centres that have already initiated the intervention. Additionally, the public health intervention is undertaken by health centre staff, who along with patients, know whether or not they have initiated the intervention. In our study, contamination between clusters, though possible, is not a major concern as TB patients and their HCs must attend the health centre of the catchment area to which their primary residence is registered.

The selection of a stepped-wedge design is considered optimal in the context of logistical, feasibility and resource challenges. However, the stepped-wedge design of a population-based implementation does not necessarily require fewer research or programmatic resources to undertake adequately compared to other study designs. In the current protocol, given a study period spanning 20 months, resources were required for monitoring the full implementation of the intervention across all centers for the entire time frame. These include study resources for data quality monitoring, training and supervision of the intervention, and data extraction from health centre charts throughout the full study period. While all operational research studies are subject to these challenges, it is a consideration in the implementation of a stepped-wedge design study where intervention cross over times occurs over a long period. If certain centres or the entire district interrupt the intervention in health centres for unforeseen circumstances, such as personnel strikes, public health outbreaks or other political challenges, then data collection, quality and completeness could suffer and at minimum require adjustment in the study time lines and addition of resources to complete data collection or worse, could lead to imbalances or biases in the overall stepped-wedge designed study.

There are several advantages and disadvantages to designing a pragmatic stepped-wedge CRCT for operational research within a routine public health program. A unique feature of the current protocol is the use of routine NTP program staff. The stepped-wedge approach using NTP personnel provides an evaluation of intervention effectiveness within actual programmatic conditions and therefore, may be more representatives then a study using strictly dedicated highly trained research personnel. The ability to sustainably integrate the intervention into the program during the study period was facilitated given the intervention was presented as a function of the routine program and not a study specific responsibility. The stepped-wedge implementation provides a useful tool to be able to logistically implement the study, allowing for targeted training within small groups as they initiate the intervention.

Public health interventions are complex and context dependent, as they are integrated within the infrastructure of existing health systems, dependent on local political, socioeconomic and cultural perspectives of the population and its public health practitioners [6]. The use of routine NTP nurses and physicians adds a complexity given that this intervention of household contact tracing is conducted within the context of numerous other responsibilities of the health centre staff. TB programs are impacted by complex treatment management protocols for active TB cases, a high burden of cases in endemic areas and numerous administrative programmatic activities (e.g., reporting forms, indicators, record keeping). Therefore, effective implementation requires planning, high quality training and active monitoring. These factors are better achieved when programs are initiated in a manageable number of sites; the stepped-wedge design allows for flexibility in determining the number of sites initiating a new program at any one time. This aspect is particularly valuable when beginning the new program in all or half of all sites would not be feasible [39]. The randomized allocation of centres to the crossover time point also eliminates preferential assignments due to either health centres’ performance evaluations or other subjective criteria. In the context of the current study, health centre NTP staff appeared motivated and engaged in training and preparation for the intervention once it is known that all centres will have to implement the new program and undergo the same intensive training, monitoring and evaluation processes, instead of some targeted added work assigned only to half of the centres.

Adherence or fidelity to the program and whether the intervention, in its intended form, is systematically applied by staff within a health centre is a distinct challenge particularly in pragmatic trials where routine program staff are undertaking the new intervention as part of their normal duties [6]. From an analytic perspective, intention-to-treat analysis (ITT) is generally the preferred analytic strategy for CRCTs, which considers outcomes based on the random allocation to the intervention arm, regardless of what happened subsequent to assignment. In practice, if there is no reported difference in effectiveness between intervention and control arms, conclusions need to consider whether the ineffectiveness of the intervention is the cause or if possible low adherence of implementers to the intervention is the more likely explanation. To examine this latter possibility, per protocol analyses, which consider actual adherence to the intervention, will also be examined in our study.

Several challenges can occur in undertaking a research study within the daily operations of a public health program, in addition to high staff turnover, worker strikes, outbreaks in other disease areas, and authorization requirements for data access. These substantial issues lead to interruptions of activities, which may not be encountered in studies using dedicated research staff. While planning and design of the stepped-wedge can help to control some of these threats, in some instances these cannot be completely avoided or controlled. The probability of an interruption occurring increase and should be expected when study periods are projected over several months within a routine operational TB program. However, dealing with unforeseen events is a common reality for most public health settings, and the evaluation of the intervention within a pragmatic setting could reflect its likely effectiveness once integrated into practice.

Stepped-wedge designs provide an important option for public health researchers and practitioners to generate intervention effectiveness data that otherwise could remain unmeasured. Typically, stepped-wedge designs are justified when feasibility, logistics and/or limited resources are important practical considerations while still allowing for a randomization process as part of the operational research. Overall, there is no indication that a stepped-wedge design requires fewer resources than other designs; the design requires resources over a longer period of time, yet involves smaller resources for training and monitoring at any given time point during the study. Finally, fidelity or adherence to the intervention may need to be considered during implementation and in the analysis, in order to correctly interpret null or negligible findings of effectiveness.

The current ongoing study will provide invaluable evidence on contextual factors that would not have been possible in traditional study designs. The findings of this study will have implications for the selection of interventions and allocation of resources in TB programming for Peru, and will be a major contribution in the field of TB prevention and TB contact tracing in LMICs.