Sustaining clinician penetration, attitudes and knowledge in cognitive-behavioral therapy for youth anxiety

Sustainment, ‘the continued use of program components and activities for the continued achievement of desirable program and population outcomes’ [1], is an understudied area within implementation science [2],[3]. Fortunately, the burgeoning of prospective implementation trials has poised the field to explore sustainment following implementation [2]. An understanding of sustainment and associated factors can be informed by an ecological perspective, such as the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework [4]. The EPIS is one of the first frameworks to delineate a separate stage for sustainment, as well as offer guidance regarding contextual factors likely to impact sustainment [4].

A seminal review of sustainment found that many naturalistic observational studies (N = 125) have been conducted following implementation of an innovation. However, the authors caution that these studies are hampered by their insufficient methodological rigor, making it difficult to draw conclusions and necessitating more empirical study [2]. Importantly, there are several areas of weaknesses in the studies conducted to date. First, and most strikingly, only 6% of the studies investigated sustainment following experimental manipulation of implementation strategies. Second, less than half of the studies reported quantitative outcomes, such as the proportion of sites or providers sustaining an EBP. Third, of those studies that did report quantitative outcomes, the majority of investigations only examined penetration [2], which refers to ‘the integration of a practice within a service setting and its subsystems’ [5]. Other important sustainment outcomes were not examined. Fourth, when examining predictors of sustainment, only 30 studies used quantitative methods, and of those studies, only 20 were guided by a conceptual framework [2]. Given these limitations, the current study contributes to the empirical literature on sustainment by: investigating sustainment following an experimental manipulation of an implementation strategy (i.e., training); examining a number of sustainment outcomes quantitatively (i.e., penetration, components of treatment utilized, knowledge, and attitudes); and examining a predictor of sustainment guided by a conceptual framework (i.e., EPIS) [4].

The primary aims were to examine sustainment for clinicians two years following receiving training and consultation in cognitive-behavioral therapy (CBT) for youth anxiety [6], an evidence-based practice [7]. This time length was chosen given that it takes two to four years for programs to be institutionalized [8]. We selected penetration, the self-reported percentage of anxious youth treated with CBT [2], as our primary sustainment outcome given that it is an important implementation outcome [5] and to build upon the previous literature [2]. In an effort to broaden understanding of sustainment, we also examined sustainment of specific components of CBT and of two individual adopter characteristics from the EPIS framework [4]: knowledge of CBT and general attitudes regarding evidence-based practices (appeal, requirements, openness, and divergence). Although knowledge and attitudes are typically described as moderators of implementation, we were interested in examining knowledge and attitudes over time to see if initial changes in these constructs at the implementation phase maintained into the sustainment phase. Furthermore, consultation session attendance was examined as a moderator of the change in CBT knowledge and EBP attitudes across all time-points, given that consultation attendance was associated with an increase in treatment fidelity (i.e., skill; adherence) in the initial study [6].

Means and standard deviations for CBT knowledge, EBPAS scores, and penetration, as well as the number of participants at each assessment, are presented in Table 4. Means and standard deviations for CBT components are found in Table 5.

The present study examined sustainment of CBT for anxious youth two years following participation in an implementation trial using educational implementation strategies (i.e., training and consultation). Guided by the EPIS framework, the sustainment of two individual adopter characteristics (i.e., knowledge and attitudes) was also examined [4]. Time spent in consultation sessions was examined as a potential moderator of change in knowledge and attitudes over time based on previous literature. Given that participation was largely driven by individual choice, consultation participation can also be considered an individual adopter characteristic [4]. Findings indicate sustainment of penetration rates from post-consultation to follow-up although not all intervention elements were equally delivered. Therapists reported low usage rates of specific components of the treatment, specifically exposure tasks, the active ingredient of CBT for child anxiety [24]. Although knowledge scores at follow-up were significantly higher than at baseline, maximal gains in knowledge achieved by post-consultation were not sustained. Attitudinal improvements toward EBPs following training and consultation were also generally not sustained. Time spent in consultation significantly moderated changes in knowledge and attitudes toward EBPs, suggesting an important mutable target for implementation and sustainment strategies moving forward. Taken together, these results suggest important implications for future implementation endeavors.

Self-reported penetration of CBT with anxious youth was sustained from post-consultation (79%) to follow-up (83%). Fidelity was not measured at the two-year follow-up, but therapists were asked to report on components of CBT for child anxiety that they had used with anxious youth. These self-report ratings suggest that CBT may have not been implemented with high fidelity. Clinicians reported extensively implementing identification and management of somatic arousal and anxious cognitions, rewards, and problem-solving. Exposure tasks, a key component of CBT for anxiety, were implemented much less extensively. This is concerning given that the exposure tasks are critical and are viewed as core components of the intervention [24]. When considering the treatment components needed to attain fidelity, even with ‘flexibility’, [25], exposure tasks are required [26]. In light of these findings, implementation efforts using training and consultation as implementation strategies may benefit from spending additional time discussing the importance and application of exposure tasks, including how clinicians might tailor exposures within their organization. Of note, approximately 30% of the consultation sessions in the present study focused on exposure tasks, suggesting that spending more time in consultation on exposure is necessary [21], although the specific amount of time needed in training and consultation to dedicate to exposure remains unknown.

Knowledge increases throughout the training and consultation phase were not sustained at two-year follow-up, although knowledge remained higher than at baseline. In order to identify contextual factors that contribute to sustainment, and in accord with the EPIS framework, an individual adopter characteristic (i.e., time spent in consultation) was examined as a potential moderator given previous findings (see Beidas et al., 2012) [4],[6]. Greater participation in consultation resulted in greater sustainment in knowledge at the follow-up period. This finding is consistent with previously reported findings and provides further evidence of the benefits of consultation following workshop training in order to improve and sustain knowledge [6]. It is worth noting that follow-up participants spent more time in consultation than non-participants. Thus, higher than average participation in consultation within a group already evidencing greater consultation than the general group appeared to contribute to positive results, further highlighting consultation as an important educational implementation strategy.

Disappointingly, improvements in attitudes gained through the training and consultation did not sustain over time. The pattern observed across two of the four attitudinal subscales measured by the EBPAS (appeal, requirements) suggested that attitudinal gains made by participating in training and consultation were not maintained. A similar pattern emerged with regard to openness to new practices, but surprisingly, not only were gains not maintained, but clinicians exhibited less openness to new practices at follow-up when compared to baseline. Belief in the utility of EBPs exhibited a different pattern. This attitude did not change from baseline to post-training, decreased from post-training to post-consultation, and then increased to baseline levels from post-consultation to follow-up. There is little literature to ground these findings within because few studies have investigated changes in attitudes over this long of a period (i.e., sustainment). Many studies measure attitudes pre- and post-training [27], and, typically, attitudes do improve following training as observed from baseline to post-consultation in this study (see Beidas & Kendall, 2010) [28]. However, this study demonstrates that new information is gleaned through measurement of attitudes at the sustainment time-point. With regard to the curious finding that openness to new practices dropped at follow-up when compared to baseline, one explanation may be that therapists who were initially very open to new practices became aware that a significant time investment (i.e., training and ongoing consultation) is necessary to gain mastery in a new practice, making them less open to new practices in the future. It is also possible that clinicians encountered difficulty implementing exposures on their own during the follow-up period, which may have resulted in less openness to CBT given its emphasis on exposures. With regard to the finding that belief in the utility of EBPs increased from post-consultation to follow-up, it is possible that after having two years to apply CBT for youth anxiety, therapists had more time to see the potential benefits of the treatment and generalized this to other EBPs.

This study adds to the limited literature regarding the sustainment of penetration, knowledge and attitudes over time. Strengths of the study include the two-year span of time examined as well as the inclusion of an individual adopter characteristic (i.e., consultation participation) as a potential moderator of sustainment. Additionally, this study looked at numerous sustainment outcomes, including outcomes not commonly examined (e.g., knowledge and attitudes). However, limitations should be noted. Despite best efforts, only 43% of the original sample participated. A lower proportion of Hispanics/Latinos participated in the follow-up study as compared to the original study. Also, a difference in consultation session attendance was found across samples. Thus, it is possible that findings from the current study do not generalize to the full sample. Given the specific focus on CBT for youth anxiety, it is possible that findings do not generalize to other interventions [29]. Other limitations pertain to measures used, including investigator-created measures and the use of self-report.

Additionally, the current study only examined one individual adopter characteristic as a potential moderator of sustainment, whereas the EPIS framework encourages examination of numerous additional inner context factors, intervention characteristics, and outer context factors [4]. Inner context factors include intra-organizational and individual adopter characteristics, whereas outer context factors include characteristics of the service environment, inter-organizational environment, and consumer support/advocacy [4]. Also, given that participants were not randomized to amounts of consultation attendance, it is possible that attendance was confounded by motivation; participants who spent more time in consultation may have had higher motivation as well as greater knowledge and more positive attitudes. Furthermore, due to time and resource constraints, the follow-up study did not include an assessment of treatment fidelity, preventing conclusions regarding the sustainment of this important implementation outcome. Future work on sustainment is needed to address these limitations. To gain a better understanding of change over time, future investigations should study sustainment over even longer periods of time. With regard to measurement, future work should incorporate multiple outcome measures, including observational fidelity (e.g., coded therapy sessions) and client outcomes. Examination of potential moderators of sustainment, informed by an ecological framework, such as EPIS, will allow for identification of mutable targets to improve implementation and sustainment efforts [4].

The field has made gains regarding the development and implementation of EBPs [19],[30]. Despite advancements, much work is left to be done, particularly around sustainment [2],[3]. The present study adds to this limited literature on sustainment by demonstrating that although clinicians sustained self-reported penetration, they did not deliver the treatment with intended components, specifically exposure tasks. Further, knowledge and improved attitudes toward EBPs failed to fully sustain over the two-year follow-up period. However, the silver lining to these discouraging results further corroborates the importance of ongoing consultation following initial training as an important implementation strategy in order to promote and sustain knowledge and attitudes over time.