Overall results and key findings
This qualitative process evaluation found, in general, that the CALM collaborative-care intervention for multiple anxiety disorders was not overly burdensome to providers and staff, and it was relatively easy to incorporate into the clinics' routine. Satisfaction with the intervention among these respondents was generally high. Primary care providers appreciated the additional referral source and the feedback they received from the ACS and psychiatrist. The majority of informants reported seeing moderate improvements in enough patients for them to find value in the intervention. A majority of informants stated, without being prompted, they would like to see the intervention continue after the clinical effectiveness study was over. We did not see much evidence of outright "resistance" to the CALM intervention, rather more a lack of motivation/buy-in from a number of providers and some not seeing it as solving key concerns of the clinics, which served more as a challenge to sustainability than uptake during the clinical trial.
There were many important facilitators to implementation, perhaps the most important being positive attitudes about the intervention among providers (buy-in). Providers who held a pre-existing belief in the importance of recognizing and treating mental health problems in primary care, who found the intervention nonburdensome, who perceived the ACS as visible and well-liked, who valued the feedback from the clinical team, and who observed positive patient outcomes (especially reduced somatic complaints) were those who most enthusiastically supported the intervention. Other facilitators were a reliable and proximate location of the ACS's workspace, having the ACS work full-time in the clinic, "face time" for ACSs to interact frequently with providers, and the perception of a relatively high prevalence of anxiety among clinic patients. Those clinics with previous experience with an on-site mental health provider and/or collaborative-care interventionist appeared to more readily implement the intervention. It is also possible that clinics with pre-existing mental health providers attracted a greater number of patients with anxiety disorders, and therefore, those clinics might have recognized a greater benefit of the intervention.
Numerous barriers to implementation were also found. First and foremost, it was clear that not all providers bought in to the intervention. Some were infrequent users of the intervention, and some never used it at all. For some, this relative lack of use appeared to be tied to their impression that anxiety prevalence was low in their clinics. For others, the lack of use appeared to be linked to a general lack of comfort with treating mental illness or a belief that mental health should not be treated in primary care. Further, these providers did not seem to respond to (or perhaps did not attend) traditional educational sessions by experts presenting evidence on anxiety prevalence and effectiveness of collaborative-care interventions. It is possible that additional marketing of the intervention might have improved provider buy-in. However, many of the providers' attitudes might have been relatively easily fixed. Future research should test the effect of various marketing strategies in primary care clinics.
In addition, it was clear that nurses in some clinics served as key advocates and sources of referral. Yet, the study did not specifically market to clinic nurses or provide educational opportunities for them. This was an important oversight. Future studies should consider active promotion of primary care interventions with clinic nursing staff. Other key barriers to implementation were having large numbers of part-time primary care providers within a clinic, using a part-time ACS, a lack of dedicated space for ACSs, unsatisfactory communication with ACSs, and engagement challenges related to low SES and Hispanic patients. Important barriers to sustainability of the intervention were the cost of the ACS (a "deal-breaking" barrier) and space concerns.
There were not many dramatic differences in reaction to the intervention across stakeholder groups or clinic types, but a few differences did emerge. Clinic administrators/managers, who were usually those charged with finding ACS work space and overseeing clinic operations, reported the most initial skepticism about the intervention and reported experiencing the most burden caused by the intervention. The burden, however, was still mild to moderate, and administrators were generally quite positive about the intervention. In terms of clinic types, the federally qualified community health centers were perhaps at some disadvantage because more of their patients were low SES, and those patients seemed to struggle more with engagement. At the same time, providers in these clinics seemed to especially appreciate the additional referral source (a free mental health intervention for their patients). Offering CBT was also a key draw for participation as some of the providers reported very little access to CBT, even when a patient was insured. Two clinics administered by a large, regional HMO and one university-affiliated clinic had experience with collaborative-care interventions, and providers in these clinics were more likely to think that sustainability would be feasible.
Findings in context of the literature
Most of the barriers and facilitators found in this study are consistent with leading conceptual models of dissemination and implementation in healthcare and the empirical literatures supporting them. For example, numerous barriers and facilitators to implementation and sustainability reported by informants had to do with provider beliefs, attitudes, motivation, and norms, and these are central determinants of implementation in the models of Mendel et al. [
34], Greenhalgh et al. [
21], and Damschroder et al. [
36]. These "personal" determinants certainly interact with the characteristics of the innovation. Most implementation models in healthcare place a good deal of emphasis on predisposing characteristics of the innovation for implementation success. For example, Greenhalgh et al.'s [
21] model, based on empirical findings, posits that an innovation has a better chance of successful adoption when it has demonstrated clinical advantage (via research evidence), compatibility with existing practices, observability (of results), low complexity, and potential for local tailoring. These same general attributes are also reflected in the diffusion and implementation models of Rogers [
37] and Damschroder et al. [
36]. Innovation characteristics interact not only with provider/staff-level attributes but also with clinic-level attributes such as culture (norms and practices of the system) and climate (worker's perceptions of, and reaction to, the characteristics of the work environment) [
38]. We found that if the clinic culture had previous experience with collaborative care and/or making internal referrals for mental health conditions, this promoted both successful implementation and increased the perception of sustainability. This study also identified barriers and facilitators associated with how the intervention was supported "on the ground" by study and clinic staff. Central to the Promoting Action on ResearchImplementation in Health Services (PARIHS) implementation framework is the notion of facilitation, defined as "making things easier for others" [
39]. This model suggests that implementation success is maximized when there are coordinated efforts to encourage participation, promote action, create supportive systems, and monitor and feedback progress. This study suggests that clinics in which ACSs operated not only as clinicians but also as facilitators in conjunction with local champions were more successful. Lastly, this study found the major barrier to sustainability to be financial (i.e., paying for the key interventionist). Most implementation models recognize and emphasize the cost and payor factor, along with others making up the "outer context" of implementation (e.g., the socio-political climate, mandates, and incentives) [
21,
34,
36,
40].
It was apparent from the analyses that at least two "meta-themes" emerged-the first being associated with
communication. Many of the barriers and facilitators we found appear related to positive communication among the ACS, primary care staff, and study-provided psychiatrist, for example, satisfaction with communication with the ACS, nurses being in or out of "the loop," positive clinical outcomes being communicated to the providers, ACSs being part-time or full-time, face-to-face hand-offs of patients, and space (proximity to ACSs). These communication issues are certainly linked to rapport and trust, and together they form a necessary foundation for a working collaborative relationship. It is clear from these analyses that for a collaborative-care intervention to be successfully implemented, there need to be positive lines of communication created that foster rapport and trust. The second meta-theme relates to the
lack of anxiety-specific barriers and facilitators observed. Most of the barriers and facilitators were similar to those noted for depression collaborative care and perhaps other collaborative-care interventions in other non-mental health conditions [
15,
41]. The one anxiety-specific barrier mentioned was a perceived low prevalence of anxiety in some clinics. This could mean that interventions such as CALM for anxiety disorders would be amenable to being combined with other collaborative-care interventions, at least for mental health conditions, as seems to be the current thrust in some health systems [
42]. Such combinations could be more cost effective and improve the "business case." The CALM intervention already combined multiple anxiety disorders and had a companion module for comorbid depression.
Study limitations and strengths
The study has several limitations that deserve mention. As noted above, this study investigated barriers and facilitators to the implementation of the CALM intervention during a clinical effectiveness trial. The implications derived here, while contributing to accumulating findings on collaborative-care interventions, still need to be tested in future implementation trials and observational studies of real-world adoption in order to be generalizable. Further, the timings of the clinic provider and staff interventions were not uniform. They occurred during the final year of a two-year intervention. Therefore, "exposure" to the intervention and its outcomes varied considerably, and this difference could have impacted the results. Perhaps most importantly, the sampling strategy likely introduced a positive bias in that most of the participants were at least moderate users of CALM, and such users were more likely to be positively predisposed to the intervention. The investigators decided that the majority of respondents should have at least moderate experience with the intervention to be able to provide the richest feedback. To counteract this bias, we did interview providers and staff with less or no direct involvement with the CALM intervention. Also, the ACS interviews asked specific questions about provider and staff buy-in from the perspective of the clinic as a whole, which also contributed balance.
The study has several strengths that also deserve mention. Very few clinical effectiveness studies devote the time and resources to an implementation-focused process evaluation. By doing so, the investigators were able to gather useful data to help them plan for future implementation efforts and reduce the time between development of research findings and adoption in routine care settings. Further, the process evaluation was large in scope, encompassing 61 key informants from multiple perspectives across 17 clinics. The use of semistructured interview guides contributed consistency and reliability to the data collection but did not limit the flow of conversation or discovery of themes. Also, "saturation" was reached, indicating that the number of interviews per stakeholder group was sufficient to fully explore the phenomena under study.