Adaptive School-based Implementation of CBT (ASIC): clustered-SMART for building an optimized adaptive implementation intervention to improve uptake of mental health interventions in schools

The primary aim of this study is to compare the effectiveness of an adaptive implementation intervention on CBT delivery among schools versus REP alone (the control). The adaptive intervention provides schools with REP + coaching from the start and subsequently augments with facilitation for schools needing additional assistance. The primary outcome is the total number of CBT sessions delivered to students by SPs over an 18-month period. Number of CBT sessions is defined further below, and includes group and individual sessions delivered to students.

Specific CBT component delivery and whether delivery of individual or group sessions were brief (< 15 min) or full-length (≥ 15 min) will also be tracked and examined as secondary outcomes. As an exploratory outcome for this primary aim, we will also examine change in student mental health symptoms among students over the study period.

Schools will be eligible if they

A school administrator who is a principal or other senior administrator at each participating school will be asked to provide data on building-wide sociodemographics and leadership support for evidence-based practices.

School professionals identified by schools are eligible if they are

School professionals will be excluded if they have a significant illness or condition that precludes their participation in the implementation strategies, including the REP training and student identification process, coaching, or facilitation, or are unable to provide informed consent for participation in the study activities.

Replicating Effective Programs (REP) [36] will be provided to all schools and is based on Rogers Diffusion Model [63] and social learning theory [64]. REP enhances EBP uptake by customizing interventions to fit the needs of specific settings through EBP packaging (tailoring of the modular CBT manual in user-friendly language), didactic training, and ongoing technical assistance provided by the TRAILS program. The package includes an overview of CBT core components, agendas describing how each component is delivered within a session, sample student screening forms, talking points for students, and suggestions for school-based delivery. REP training to be provided by TRAILS covers modular CBT core components including screening and identification of students. REP technical assistance consists of regular scheduled conference calls during which SPs may receive support from an expert CBT clinician and open access to an interactive website that provides additional resources (e.g., video demonstrations, case simulations) (Table 1).

The coaching implementation strategy (Table 2) is provided by TRAILS clinicians and is derived from the school-based Positive Behavior Interventions and Supports (PBIS) model of coaching for individual development [65]. Coaching uses a CBT training expert to attend in person to observe group sessions led by the SPs, provide live feedback [66, 67], and model the use of core CBT elements to improve SP competence [65, 68‐72]. All SPs from schools randomized to coaching in phase 1 will receive weekly visits from a CBT coach for a minimum of 12 weeks, which will occur in the context of the SPs weekly CBT group. After 12 weeks of on-site coaching, SPs are evaluated on their CBT skill delivery through a short objective competency quiz. SPs deemed to need a full second round of coaching based will receive another full 12 weeks of coaching.

Facilitation (Table 3) is based on the Integrated-Promoting Action on Research Implementation in Health Services Framework [73] and promotes provider self-efficacy [74] in mitigating organizational barriers to EBP adoption. Facilitation is delivered via regular phone contact for at least 10 weeks with the SPs by an expert in school and mental health care organization, implementation methods, and use of CBT and EBPs in schools. The facilitator will support SPs in strategic thinking and leadership skills to address organizational barriers covering the following:

A longitudinal survey will be given to consenting school administrators to record percentage of students eligible for free/reduced lunch, average classroom size, attendance rate, number of students referred to psychiatric emergency services, and administrator tenure. Administrators will also complete the Implementation Leadership Scale (ILS) [78] to assess institutional support for EBP. No identifying information will be collected as part of these assessments, and no compensation will be provided. Administrators will also be asked to provide approval for participating SPs to collect academic indicator data on GPA, absences, suspensions, and expulsions for participating students.

SPs will also complete longitudinal web-based surveys that include demographic background, level of education, job tenure, prior experience administering CBT, and knowledge and perceptions of CBT delivery using the CBT Knowledge Questionnaire [79]; Provider Attitude Survey [80]; Treatment Manuals Survey [81]; and the Psychotherapy Practice Scale [82]. SPs will be compensated for all completed assessments. SPs will also complete the ILS to ascertain leadership support, and two other validated measures related to support for EBPs—the Implementation Climate Scale (ICS) [83] and the Evidence-Based Practice Attitude Scale (EBPAS) [84].

Consistent with real-world fidelity monitoring for quality improvement purposes [85], the abovementioned web-based SP weekly assessment will be used to track number of CBT sessions delivered and CBT content delivered each week.

SPs wil be encouraged to identify 10 students that they believe could benefit from CBT prior to and during CBT training. SPs will be trained to create a mini-registry of students using a web-based instrument designed by TRAILS to communicate with other Qualtrics surveys (see Additional file 2: Appendix 2). Students identified by the SP will complete secure electronic surveys on mental health symptoms and health care utilization using the previously described web-based tool (Table 3). SPs will provide to the student in person an information sheet outlining the study eligibility requirements, assessments, compensation, and risks and benefits. A waiver of documentation of consent and waver of parental consent was obtained for ascertaining student outcomes from local IRBs. SPs will be required to provide students with a private location for completing all assessments and will reassure students prior to each assessment that all answers will be de-identified and that they will not have access to the responses. Measures will include student sociodemographic characteristics, health behaviors (e.g., substance use), CBT receipt, knowledge, and use of CBT skills, mental health symptoms (PHQ-9T, GAD-7), and access to mental health services and other healthcare use (e.g., ED referrals or admission). To ensure that students are not coerced into participation, they will be asked to confirm on the web-based survey that they would like to submit their answers. SPs will not be informed if students opt to not submit their answers after completing the survey. Students will be compensated for each survey completed over the 15-month period. In order to protect student privacy from the study staff, SPs will facilitate all student compensation.

For each implementation strategy, we will calculate the average costs and average outcomes per SP using methods described elsewhere [39]. The primary implementation costs are the personnel time spent in REP activities (e.g., SP training, TA), coaching (e.g., time to hire/train coaches, network maintenance, SP coaching time), and facilitation by study participants (including SP and school administrator time). Costs will be quantified as hours multiplied by wages and fringe benefits for each person. Wage rates will be obtained from school records, and in cases where this information is not available, average wages for each occupational level will be used from the Bureau of Labor Statistics. Hours will be tracked through attendance logs for each implementation activity.

To assess costs of delivering CBT, 40 randomly selected SPs will also be asked to complete time-motion surveys for 2 weeks (starting 4 weeks after the phase 2 randomization) that ask about time allotted to providing CBT versus other forms of student counseling, care, or crisis management. School services will be translated to costs based on the wage rates of school providers.

Student-level service costs of CBT delivery and other use will also be estimated from study records of participation in CBT sessions, academic indicators, and self-reported utilization survey data on inpatient, emergency department, and outpatient use outside the school setting. Health care costs will be assigned using Current Procedural Terminology (CPT) codes, and a relative value unit (RVU) weight in the Medicaid Fee Schedule calculates standardized costs in US dollars for each service adjusted for annual levels of inflation using the consumer price index.

The primary aim analysis will determine the effect of the most intensive adaptive implementation intervention, by comparing the total number of CBT sessions delivered by SPs over the course of 18 months between schools receiving REP alone (the control) versus schools receiving the adaptive intervention (REP + coaching + facilitation for schools that are eligible).

For exploratory aim 1 analyses, incremental cost effectiveness ratios (ICERs) will be calculated for each relevant comparison of implementation interventions by dividing the incremental average costs by the number of CBT sessions delivered as well as the number of depression or anxiety-free days based on PHQ-9T or GAD-7 student score changes between each time point.

Exploratory aim 2 analyses will assess whether the implementation intervention effectiveness is moderated by SP or school-level factors including SP prior training and baseline perceptions of CBT, as well as perceived school administrator support for adoption of CBT. Results of these analyses will be used to construct a more deeply tailored adaptive implementation intervention that further improves uptake, and particularly SP delivery of CBT.

Exploratory aim 3 analyses will test mechanisms through which the coaching and facilitation implementation strategies increase frequency of CBT delivery and/or improve student mental health outcomes.

Missing outcome data may occur due to school or SP dropout or loss of contact with SPs or students. Our sample retention protocol will ensure that all efforts are made to obtain primary outcome measures for all SPs in all 100 schools. For our primary SP-level outcome, based on preliminary data from TRAILS, we anticipate an attrition rate of < 10%. Prior to conducting all primary and secondary data analyses, missing data will be dealt with explicitly using multiple imputation methods for SMART studies [86, 87].