Testing the leadership and organizational change for implementation (LOCI) intervention in substance abuse treatment: a cluster randomized trial study protocol

LOCI is framed within the Exploration, Preparation, Implementation, Sustainment (EPIS) framework [21], focusing primarily on the inner organizational context and the Preparation and Implementation phases. As shown in Fig. 1, LOCI takes an active approach to improving leadership and congruent organizational strategies that lead to improved transformational and transactional leadership, implementation leadership, and subsequent implementation climate and psychological safety climate. These, in turn, are hypothesized to lead to changes in provider attitudes toward EBP, implementation citizenship behaviors, and to better EBP fidelity and implementation process [4, 35‐38]. Consistent with the EPIS conceptual framework, LOCI creates change at multiple levels within a provider organization (e.g., executives/mid-managers, workgroup supervisor, service provider) to foster a context supportive of EBP implementation and sustainment.

It is important to address leadership at the appropriate organizational levels. “First-level” leaders manage and supervise those providing direct services to clients and can be particularly important in influencing staff perceptions and behavior. Workgroup supervisors are first-level leaders and are important in effective workgroup functioning [39], yet public sector first-level leaders are often promoted from clinical or service positions without adequate training in leadership [13]. SUDT workgroup supervisors are frequently responsible for implementing innovations and meeting administrative and productivity requirements [40] but can be organizational “change agents” to inspire and motivate followers to implement change [41, 42]. They play a critical role in staff perceptions of support for using an innovation such as EBP [43]. Clinical workgroup supervisors are likely to be more effective with the buy-in and support of middle and upper organizational management [44‐47]. It is important to consider multiple levels within provider organizations when working to create a context supportive of EBP implementation and sustainment [21, 48‐50]. LOCI entails developing organizational strategies tailored to support first-level leaders while facilitating buy-in and support from upper and mid-level managements for the strategic goal of effective implementation through the creation of a positive EBP implementation climate. In LOCI, members of the research team support workgroup supervisors and management to develop a set of strategies to embed an EBP implementation climate and support EBP implementation with fidelity [2, 51].

Motivational interviewing (MI) is a leading EBP for SUDT. MI was developed after a series of findings illustrated that therapist empathy during treatment, in contrast to the treatment intervention itself, accounted for a larger proportion of variance in SUDT outcomes including post-treatment relapse [52]. MI emphasizes using an empathic client-centered clinical style while evoking and strengthening the client’s own verbalized motivations for overcoming ambivalence about change [53]. In the spirit of MI, it is the patient’s role, as opposed to the clinician’s, to articulate and resolve their own ambivalence regarding change. Treatment providers achieving high levels of MI fidelity are informative, supportive, respectful, and collaborative; their patients are more satisfied, more likely to be retained in care, more committed to treatment regimens, and have better outcomes when compared to those whose providers do not use MI [54‐64].

With NIMH support (Grant No. R21MH082731), experts in leadership and implementation from business and management schools were engaged to work with the investigators and mental health program managers (i.e., first-level leaders) and an instructional design consultant to develop the training strategy and materials. Next, a pilot study involving 12 mental health program managers, randomly assigned to LOCI or a leadership webinar, was conducted. Results demonstrated feasibility, acceptability, perceived utility, and change in leadership behavior [1, 23, 65].

There are two main potential impacts of the proposed study. First, improving leadership and organizational climate for EBP will advance both implementation science and will provide an empirically tested implementation strategy to decrease the lag between intervention or innovation development and deployment and sustainment in usual care settings [66]. Demonstrating the efficacy of LOCI will provide service systems and health care organizations with a way to simultaneously improve leadership and institute organizational strategies to support and promote effective EBP implementation and sustainment. Second, although counselors and clinicians may receive training in MI, leaders and organizations often do not apply strategies to effectively implement MI or support staff to deliver MI with fidelity beyond initial training [67, 68].

To our knowledge, there are no other empirically tested workgroup level approaches that bring together leadership and organizational strategies to improve strategic climate for EBP and implementation effectiveness. In addition, this proposed study utilizes new reliable and valid measures of implementation leadership [23], implementation climate [30], and implementation citizenship [69] and will apply the Stages of Implementation Completion (SIC) measure [37] to assess LOCI and MI implementation process.

Multiple study designs were considered by the research team in consultation with collaborating agencies. In order to reduce participant burden, a multiple cohort design, cluster randomized trial that randomizes workgroups within agency to either LOCI or a webinar control was selected. Allocation was determined by the statistician on the project. Three consecutive cohorts of participants will be included. The LOCI intervention lasts for 12 months and the webinar control intervention takes place in months 1, 5, and 8 for each cohort. The unit of analyses will be workgroup, randomized by site within agency with care taken that co-located workgroups are assigned to the same condition to reduce the chances of contamination.

The study will take place with workgroups from SUDT agencies in California, USA. California is home to 38 million individuals in 58 counties encompassing urban and vast rural areas. The population is diverse (39.4% non-Hispanic White, 38.2% Hispanic, 14.4% Asian/Pacific Islander, 6.6% Black, 1.7% American Indian). A language other than English is spoken in 38.5% of households. Education levels are 80.8% high school graduate and 30.2% bachelor’s degree or higher, and 14.4% of persons have income below the poverty level.

Sample demographics for service providers and workgroup supervisors are expected to be approximately 65% female, 58.7% Caucasian, 27.4% Latino, 18.5% African-American, 2.6% Asian American/Pacific Islander, 2.1% Native American, and 18% “other.” Regarding education, we estimate 18.1% of staff will have a master’s degree, 28.7% college graduates, and 38.8% completed some college level coursework. The proposed settings and providers may also address clients’ co-occurring mental health problems. Consent forms will be available in English and Spanish, and there will also be the capacity for MI coding in both English and Spanish.

Data are to be collected at baseline (prior to the interventions), 4, 8, 12, and 16 months (4 months after the end of the LOCI intervention). Organizational data will be collected from all participants (service providers, workgroup supervisors, and executives/mid-managers) via web surveys and downloaded into a database programmed for error identification and checking. Each type of respondent will complete a set of measures in their survey, and not all measures will be collected at each time point. The provider/supervisor surveys will take about 20–30 min to complete, whereas the survey for executives/mid-managers will take about 5–10 minutes per supervisee. Participants will be compensated with a $25 electronic gift card via email for completing each survey. A portion of the data collected in these surveys (i.e., FRL, implementation leadership, EBPAS, EBP implementation climate) will be shared with supervisors in the LOCI condition as a part of the leadership development intervention. Data from managers/executives will be shared with the supervisors, and this is comparable to a typical performance review. Data from service providers will be shared with their supervisors in an aggregate manner to protect the confidentiality of individual provider responses. All data will be stored on a secure server with incremental and weekly full back up.

The study will obtain estimates of the leadership intervention effect by comparing multiple intervention measures on provider-rated leadership, workplace climate, work attitudes, and attitudes toward adopting and implementing EBPs. The workgroup will be the unit of analysis, with sites randomized within agency to ensure that the 5% of co-located workgroups will be randomized into the same condition to avoid contamination. For 95% of workgroups, workgroup is synonymous with site. We will stratify by the number of workgroups per site to ensure equal workgroups in each condition.

Data analytic strategies will follow the recommendations of Brown et al. [85] and the Prevention Science and Methodology Group for randomized field trials [86]. Preliminary data screening and cleaning will require examination of data distributions for normality and missing data patterns at both the univariate and multivariate levels. Once complete, hierarchical linear modeling (HLM) [87] will be used as the primary statistical model due to the nested structure of the data: repeated measures (time) (level-1) nested within service providers (level-2) nested within workgroups (level 3). Because randomization is at the workgroup level, and due to the statistical complications associated with the small number of agencies and a number of workgroups being located at the same site, dummy-coded agency and site variables will be entered at the level of the workgroups for all analyses, which implicitly controls for agency and site effects [88]. Random effect variance will be evaluated for the remaining levels of the nested data structure. In the presence of near zero estimates of variance estimates, terms will be removed from the statistical model.

HLM analyses for aims 1 and 2 will test specific intervention effects (LOCI vs. webinar control condition) on target outcomes (e.g., leadership behaviors, implementation climate). Of primary interest in these analyses is the Time (level-1) × Intervention Status (level-3) cross-level interaction effect. If the cross-level interaction term(s) are statistically significant, follow-up examination of these effects will follow the procedures of Preacher, Curran, and Bauer [89]. Implementation of cross-classified models will be explored given that some shifting or personnel may occur during the course of the study [90].

Aim 3 will test fidelity. The statistical methods used will be based on the HLM models described for the first two aims; however, since measurements for this outcome are averaged over time, there will not be a modeled covariance structure. For count outcomes related to implementation process (workgroup level), we will use a negative binomial regression model instead of a Poisson regression model to allow for the possibility of over-dispersion.

Finally, mediational effects described in aim 4 will use procedures for multilevel data structures outlined in MacKinnon [91]. For all analyses, covariates will be considered at both the provider and workgroup level. Inclusion of confounder variables at the provider level is important given that randomization to LOCI vs. control condition will not occur at this level. Inclusion of covariates at the workgroup level will adjust for chance imbalances and increase the precision of the analyses.