Skip to main content
Erschienen in: Implementation Science 1/2017

Open Access 01.12.2017 | Study protocol

Improving and sustaining delivery of CPT for PTSD in mental health systems: a cluster randomized trial

verfasst von: Shannon Wiltsey Stirman, Erin P. Finley, Norman Shields, Joan Cook, Rachel Haine-Schlagel, James F. Burgess Jr, Linda Dimeff, Kelly Koerner, Michael Suvak, Cassidy A. Gutner, David Gagnon, Tasoula Masina, Matthew Beristianos, Kera Mallard, Vanessa Ramirez, Candice Monson

Erschienen in: Implementation Science | Ausgabe 1/2017

Abstract

Background

Large-scale implementation of evidence-based psychotherapies (EBPs) such as cognitive processing therapy (CPT) for posttraumatic stress disorder can have a tremendous impact on mental and physical health, healthcare utilization, and quality of life. While many mental health systems (MHS) have invested heavily in programs to implement EBPs, few eligible patients receive EBPs in routine care settings, and clinicians do not appear to deliver the full treatment protocol to many of their patients. Emerging evidence suggests that when CPT and other EBPs are delivered at low levels of fidelity, clinical outcomes are negatively impacted. Thus, identifying strategies to improve and sustain the delivery of CPT and other EBPs is critical. Existing literature has suggested two competing strategies to promote sustainability. One emphasizes fidelity to the treatment protocol through ongoing consultation and fidelity monitoring. The other focuses on improving the fit and effectiveness of these treatments through appropriate adaptations to the treatment or the clinical setting through a process of data-driven, continuous quality improvement. Neither has been evaluated in terms of impact on sustained implementation.

Methods

To compare these approaches on the key sustainability outcomes and provide initial guidance on sustainability strategies, we propose a cluster randomized trial with mental health clinics (n = 32) in three diverse MHSs that have implemented CPT. Cohorts of clinicians and clinical managers will participate in 1 year of a fidelity oriented learning collaborative or 1 year of a continuous quality improvement-oriented learning collaborative. Patient-level PTSD symptom change, CPT fidelity and adaptation, penetration, and clinics’ capacity to deliver EBP will be examined. Survey and interview data will also be collected to investigate multilevel influences on the success of the two learning collaborative strategies. This research will be conducted by a team of investigators with expertise in CPT implementation, mixed method research strategies, quality improvement, and implementation science, with input from stakeholders in each participating MHS.

Discussion

It will have broad implications for supporting ongoing delivery of EBPs in mental health and healthcare systems and settings. The resulting products have the potential to significantly improve efforts to ensure ongoing high quality implementation and consumer access to EBPs.

Trial registration

NCT02449421. Registered 02/09/2015
Abkürzungen
BIA
Budget impact analysis
CDCEQ
Clinician Demographic Characteristics and Experience Questionnaire
CPT
Cognitive processing therapy
CQI
Continuous quality improvement
CROMIS
Client Reported Outcome Monitoring Information System
DOD
Department of Defense
DSF
Dynamic Sustainability Framework
EBP
Evidence-based practice
EBPAS
Evidence-based practice attitudes scale
FID
Fidelity
ICA
Implementation climate assessment
ICC
Intraclass correlation
ILS
Implementation Leadership Scale
IS
Implementation strategy
LC
Learning collaborative
LOI
Learning Organization Inventory
LOS
Learning Organization Survey
MHS
Mental health system
OSC
Organizational social context
OSI
Operational stress injury clinic
PCIS
Perceived Characteristics of Intervention Scale
PCL
PTSD checklist
PDSA
Plan, Do, Study, Act
PE
Prolonged exposure
PTSD
Posttraumatic stress disorder
QRCPS
Quality-rated CPT provider status
TX
Texas
US
United States
VA
Veterans Affairs
VAC
Veterans Affairs Canada

Background

Thousands of public sector clinicians have been trained to deliver evidence-based psychotherapies (EBPs) due to recent mandates and investments in implementation [1, 2]. Patients treated during EBP training programs experience substantial symptom improvement [35]. Despite this progress, efforts to implement and sustain EBPs in these systems face significant challenges. One finding is that penetration (integration of a practice within a service setting and its subsystems) is low [6, 7]. Veteran Affairs clinicians rarely cite established contraindications as reasons not to offer EBPs; [8, 9] instead, they cite challenging symptom profiles, a need for more consultation, and clinic-level barriers [10, 11]. A second challenge is that research suggests that fidelity (adherence to prescribed elements of treatment and competence/skill of delivery) [12] may be low when these interventions are implemented [13, 14]. Challenges in implementing EBPs lead to adaptation without systematic efforts to understand the impact on symptoms and other outcomes [15, 16]. While some adaptation and latitude in EBP delivery may be appropriate [17], effective [1821], or promote implementation [22], other adaptations result in discontinuation of core elements [15], integration of non-evidence-based strategies [13, 23], and worse outcomes [24, 25]. In addition to these challenges, local capacity can impact sustainability and quality of delivery. A significant minority of clinician trainees do not meet established EBP competency criteria but deliver EBP protocols or EBP elements in their practice [11, 13, 15]. Turnover and organizational context can also impact sustained EBP delivery [26, 27]. Patient access and clinical outcomes may suffer as a result [23, 24, 2830]. Thus, in the context of routine care, improvement, rather than maintenance, [31] of clinical outcomes may be an appropriate goal. Finally, while EBPs are cost-effective [32], the budget impact of supporting implementation can be a barrier [3335].
Failure to provide EBPs to those with PTSD can have a significant public health impact. Consequences of inadequate treatment include risk of suicide, overuse of healthcare, work absenteeism, reduced productivity, unemployment, and family disruption [36, 37]. Cognitive processing therapy (CPT), which has been implemented in at least eight countries and in mental health systems throughout the world, has a strong evidence base and is effective with a variety of patient populations [19, 20, 3842]. The current study will aid in identifying strategies to promote sustainability of CPT and improve patient-level outcomes, thereby informing efforts to implement EBPs for psychiatric disorders more broadly.
Researchers have identified serious gaps in knowledge regarding best practices for improving and sustaining EBPs in routine care [4345]. There have been few, if any, experimental investigations of strategies to sustain EBPs. Few studies on sustainability have assessed a full range of the key outcomes, such as patient-level mental health outcomes [46]. To address these gaps and identify effective strategies for sustaining EBPs in routine care, we seek to compare two promising yet different implementation strategies (ISs), fidelity-oriented consultation, and continuous quality improvement.

Proposed interventions

Existing research and theory suggest two contrasting ISs, one oriented to fidelity and the other to mutual adaptation and quality improvement. These strategies have potentially different mechanisms, costs, and relative benefits that could impact their fit in differing contexts.

Fidelity-oriented learning collaborative (FID)

Studies show that without consistent follow-up or ongoing support of clinicians to promote fidelity after initial training, the training effects quickly dissipate [47, 48]. Although a recent meta-analysis demonstrated no overall link between observer-rated fidelity and symptom change across a range of treatments and disorders, when analyses were conducted to avoid the potential temporal confound between fidelity and symptom change and establish the temporal precedence of fidelity, two aspects of fidelity, adherence to the protocol and competence (skill of delivery) in early CBT sessions were associated with subsequent decreases in depression [49]. Later research suggested that fidelity to the key aspects of CPT, as opposed to prescribed session elements, was associated with symptom change [50]. Other studies demonstrated patient-level benefits of fidelity-oriented support for EBPs as compared to general professional development-oriented support [5153]. A FID strategy has been shown to lead to clinician achievement of benchmark levels of fidelity [54]. We would thus expect FID to impact clinical outcomes through CPT fidelity as a mechanism of change. Fidelity support also appears to impact other implementation outcomes, most notably sustained EBP capacity, delivery [55], and support activities [56]. Fidelity support has also been associated with lower staff turnover, which improves workforce capacity [53].

Continuous quality improvement-oriented learning collaboratives (CQI)

Organization-level barriers to implementation are commonly found in routine care settings and may impact sustainability [4, 26, 57, 58]. In our study on CPT training consultation in Canada, [59] in Texas, and in the US VA [10, 11], clinicians cited organization-level challenges to delivering EBPs for PTSD and hesitance about offering EBPs, due in part to a perceived lack of fit between the treatment and the patients [9]. These problems may contribute to the low rates of penetration [6, 7]. The Dynamic Sustainability Framework (DSF) suggests that the dynamic “fit” between an intervention and its delivery context is critical to sustainability [60]. It rejects the assumption that deviation from the protocol leads to decreased benefit and advocates for mutual adaptation and continuous refinement of EBPs in real world contexts [60]. CQI, identified in the DSF as a facilitator of sustainability, has been used successfully in healthcare and mental health settings [61, 62] and has guided EBP adaptation [63]. CQI also fosters learning organizations, which are more likely to improve care and innovate [64]. We therefore expect that the mechanisms by which CQI impacts clinical outcomes are EBP adaptation and functioning as a learning organization.
While high-level leadership support is strong, and structural and policy level changes and investments have been made to support implementation within many systems, [2] the individual clinicians who must ultimately deliver treatments have identified idiosyncratic challenges at local levels. The locally oriented, clinician-led approach of CQI-oriented learning collaboratives [61] (LCs) may facilitate sustainability, but this possibility remains unexamined [65]. On the other hand, greater time, cost, and personnel burden [66, 67] may be a barrier to CQI, and whether the CQI process and resulting adaptations lead to more effective treatment and implementation is unknown. To answer these questions, we will conduct a mixed method, type-III hybrid design, which allows simultaneous study of ISs, their mechanisms, and clinical outcomes [68].

Summary of the proposed project

Clinics in three mental health systems that have implemented CPT for PTSD will submit baseline session recordings and patient data for at least two patients (n = 192) before random assignment to either 12 months of FID (n = 16 clinics, 48 clinicians, 192 patients), or CQI (n = 16 clinics, 48 clinicians, 192 patients). Outcomes will include patient self-reported PTSD symptom outcomes (primary); independent fidelity ratings; and penetration, adaptation, and capacity to deliver CPT [46, 69]. We will also investigate engagement in, credibility and costs of, and satisfaction with each IS [mechanisms by which the interventions impact patient outcomes] as well as contextual factors that may impact sustainability. The mixed method design will include qualitative methods for a richer understanding of the process and outcomes for each of the ISs [70]. This study will capitalize on infrastructure created for a previous trial comparing EBP consultation strategies [59] as well as existing infrastructure in the participating mental health systems and strong collaborative relationships and experience with each participating mental health system. The study has the potential to advance implementation science beyond observational studies of EBP sustainability by providing much-needed information on the effectiveness of interventions to promote long-term EBP implementation.

Methods

Summary

We plan to randomize clinics and associated clinicians in three mental health systems that have implemented CPT for at least 2 years into one of two conditions: CQI-LC or FID. Implementation outcomes will be assessed. ISs will be compared over the course of the 1-year intervention phase and a 1-year follow-up. Mechanisms of action and budget impact will also be explored. A phased rollout of the ISs will allow baseline data to serve as within and across subject comparison data. Potential individual and contextual influences on sustainability will also be assessed through surveys and interviews to elucidate how these factors may impact CPT delivery and IS engagement and outcomes.

Recruitment

Participating clinics and clinicians will be drawn from the three mental health systems: the VA Canada Operational Stress Injury (OSI) National Network and affiliated clinics (VAC), the United States VA system, and the State of Texas (TX). Each of these systems has implemented CPT in the past several years, with standardized training across the networks which includes an initial workshop followed by 6 months of consultation. As Table 1 indicates, the systems are diverse in terms of clinician backgrounds, patient populations, and financing.
Table 1
Characteristics of participating mental health systems
Source
No. of clinicians available for recruitment
No. of clinics (planned/ available)
Clinic type/population
Clinician degrees
Payment/reimbursement
Penetration
VA Canada OSI Network (VAC)
134 OSI clinicians and contracted community clinicians, 65% received CPT provider status
11/38
VAC clinics and community-based/veterans and civilians
M.A., PhD/PsyD
Government veteran benefits, private insurance, self-pay for civilians
~60% of eligible patients (by self-report)
United States VA system (VA)
2996 trained, 73% reached provider status
11/138
VA hospitals and clinics/veterans
PhD/PsyD, MSW, MD
Typically VA benefits, TriCare
112–20% eligible patients [6]
Texas (TX) Community MH Clinics
253 trained/53% reached CPT provider status
10/115
Civilians and rural veterans
MSW, LICSW, LPC, CART, MA, PhD/PsyD
Medicaid, VA Tricare, self-pay, private insurance
To be assessed
We plan to recruit and enroll an average of 3–4 clinicians per clinic across 32 clinics over the first 2 years of the study.

Inclusion/exclusion criteria

We will recruit VAC and VA clinics, TX, and Canadian community clinics that participated in CPT training and have at least 4 clinicians eligible to participate. Clinics are defined as work units under the same supervisor or service line within an organization. Clinicians and supervisors who maintain a caseload will be eligible to participate in the study if they (1) are CPT-trained and provide psychotherapy to individuals with PTSD, (2) are willing to record therapy sessions, provide work samples for baseline assessment, and complete baseline surveys, and (3) have internet access. We considered only enrolling clinicians who had achieved a certain level of CPT adherence (e.g., “provider status”) but recognized that stakeholders need to understand how to improve EBP delivery among all clinicians who deliver EBPs within the system, including the 23–47% of clinicians (Table 1) who receive training but do not achieve provider status. Their inclusion allows assessment of ISs’ ability to improve capacity to deliver high quality CPT.
Eligible patient participants are Veterans or non-Veterans (1) diagnosed by their clinicians to have current PTSD, with a minimum PTSD checklist (PCL) score of 33 (based on the most current validation and cutoff scores that indicate a likely PTSD diagnosis for the PCL-5) [71] 2) who have not previously received CPT, and (3) are willing to consent to completing symptom measures and have their sessions audio recorded and reviewed. Patients are permitted to continue other psychotherapies if the treatment is not specifically focused on PTSD. Ineligible patients are those ineligible for CPT based on the state of research evidence as follows: (1) current uncontrolled psychotic or bipolar disorder, (2) [past month] substance dependence requiring daily use or detoxification (eligibility can be revisited if individuals enter recovery), (3) imminent suicide/homicide risk requiring immediate intervention, and (4) cognitive impairment precluding therapy engagement (e.g., significantly impaired memory or attention).

Randomization and timeline

Randomization will occur at the clinic level. Given the relatively small number of clinics, a stratified randomization procedure will be used to match the ISs (after baseline) on size (# of clinicians in each clinic) and system. The statistician will create a block for each combination of covariates and assign each clinic to the appropriate block. Simple randomization will occur for clinics within each block. As illustrated in the timeline, ISs will occur in three waves, and within each wave, clinics will be randomized further into immediate or delayed (6 months) start. This will facilitate a comparison with no IS [72] using symptom and session data collected from the baseline phase of the delayed start group. Administrative data from before and during the IS phase will facilitate further comparison of symptom change for patients seen before vs. during ISs. During baseline and to keep the delayed start group engaged, we will offer webinars to introduce study technology and provide general updates on CPT research findings (with no guidance for actual practice). Incentives for engagement in baseline include written fidelity feedback from one baseline session after the IS phase starts, clinician and clinic-level gifts for providing baseline data, and CE credits for webinars.

ISs

PracticeGround, an internet-based EBP training and learning community platform, will be used as the platform for IS meetings in this study. It has capacity for usage metrics and provision of CE credits. Previous research demonstrated its effectiveness as a platform for EBP training [73, 74]. A full range of materials for CPT online support (e.g., webinars, video clips, presentations, and handouts on CPT-related topics) have been integrated into PracticeGround.
Each IS cohort will include a maximum of eight clinicians across up to two clinics within the same system. After a 2-hour web-based learning (for CQI) or booster (for FID) sessions, they will meet biweekly for the first 3 months, then monthly over the last 9 months of the [12-months] active intervention phase, with between-meeting message board correspondence and communication within individual clinics. We will review recordings to assess adherence to the FID or CQI manual and differentiation of ISs with a checklist based on a review of CQI elements and the key IS elements identified in the manuals [75] and will track correspondence within each condition to compare communication.

Fidelity-oriented consultation (FID)

FID will be led by a CPT expert using structured format based on the effective standard CPT consultation model [4, 5, 76]. One hour meetings will consist of fidelity (adherence and competence) feedback based on discussion of cases and session audio review during the meeting, with guidance on challenges to CPT fidelity, e.g., how to use CPT without deviation from the manual. Participants will also review CPT training modules to improve fidelity. Facilitators will be certified CPT consultants. PDSA cycles will not be used.

CQI learning collaborative (CQI)

CQI facilitators have been trained in CQI and LC strategies by study team members who have experience in facilitating CQI-oriented LCs. The framework for the CQIs is based on the Institute for Healthcare Improvement’s Breakthrough Series collaborative model for quality improvement [77], which has been used successfully in mental health [61]. Prior research indicated that in-person learning sessions do not add a significant value relative to associated costs [78], so initial learning sessions will be trainings in CQI principles that will occur on PracticeGround. Action periods take place between subsequent hour-long Web-based meetings, allowing time for each team to implement change ideas identified during meetings and study the effectiveness of those change ideas using the Plan-Do-Study-Act (PDSA) cycle of learning. Examples of goals might include increasing CPT engagement, improving effectiveness for individuals with particular symptom profiles, or advocating for more frequent sessions within the clinic. Consistent with the CQI model, non-clinicians who are the key to producing change can participate in the CQI meetings to facilitate change.

Hypotheses and assessment strategy

We propose to assess the impact of CQI and FID on a variety of the key implementation outcomes with the following hypotheses:
1.
Patient symptom improvement
(a)
Our primary outcome, change in PCL-5 scores, non-inferior for CQI as compared to FID
 
(b)
Outcomes for ISs will be superior to outcomes for patients seen during the no intervention phase
 
(c)
The ISs will impact change in symptoms through different mechanisms (increased fidelity for FID [30, 79], CPT adaptation, and development of learning organizations for CQI) [64]
 
 
2.
Cost, as measured by budget impact, will be greater for CQI, which requires increased personnel burden (management involvement and activities between meetings) [61], than FID.
 
3.
Fidelity (adherence and competence) will be greater for FID, which targets fidelity [54], than CQI [60].
 
4.
Adaptation (# of changes to CPT) will be greater for CQI than FID due to planned adaptations in CQI [60].
 
5.
Penetration (% of eligible patients who receive CPT) will be greater for CQI than FID, as CQI targets barriers to patient enrollment and retention and allows adaptations to improve fit [60].
 
6.
Workforce capacity (% of clinicians who receive CPT provider status) will be greater for FID, which is intended to improve skills and promote adherence [54], the benchmarks for quality-rated provider status.
 
7.
Organizational capacity (learning organization status and implementation climate) will be greater after CQI, which facilitates development of learning organizations [64] and reduces barriers to implementation [60, 80].
 

Patient measures

To examine hypotheses 1A and B, we will use the posttraumatic stress disorder checklist (PCL-5), a validated 20-item self-report measure that assesses the 20 DSM-5 symptoms of PTSD [71] that patients routinely complete in CPT. Each item is measured on a 5-point Likert scale. These are recorded in VA clinical records and can be extracted at the clinic and patient level. VAC collects the PCL-5 in their secure, Web-based Client-Reported Outcome Monitoring Information System (CROMIS), which is based on the OQ outcomes monitoring system [81]. For CPT patients, PCLs are done at pre-treatment, at every CPT session, at post-treatment and 3-months post-treatment, for a total of 12–15 administrations.

Additional patient measures

Patient demographics and diagnoses will be collected from clinicians and used as covariates in analyses. Patients will also complete The Outcome Questionnaire-45 (OQ-45), a validated, 15 min measure of functioning and other key outcomes of interest (symptoms, interpersonal problems, social role functioning, and quality of life) that are of interest in mental health [82, 83] (baseline, mid- and post-treatment) and the Patient Health Questionnaire (PHQ-9) [84] to assess symptoms of depression (at every session). They will complete a post-treatment Client Satisfaction Questionnaire (CSQ-8) [85] to assess satisfaction with treatment at the end of treatment.

Cost (budget impact)

To test hypothesis 2 and lay the groundwork for future, more comprehensive economic evaluations, we will use a framework for best practices for budget impact analysis [86].

Clinician CPT fidelity assessment

Observer-rated fidelity measures

We will assess fidelity for hypothesis 3 and as a potential mechanism of change (hypothesis 1C) via observer ratings of a randomly selected subset of session recordings. The CPT fidelity measure (Nishith and Resick Cognitive Processing Therapy Fidelity Scale. Unpublished Manuscript) examines clinicians’ adherence and competence to specific CPT interventions prescribed in each session. Clinicians are rated on their adherence to the protocol (i.e., the degree to which they performed a particular task, on a 0–2 Likert-type scale) and competence in delivery of these elements (rated on a 7-point, Likert-type scale). Sessions will be continuously uploaded by clinicians to a secure server (using procedures that were successfully used with geographically dispersed clinicians in our consultation study) [62, 87] and randomly selected for rating to minimize bias that may occur if clinicians were instructed to only provide sessions at designated time points. Rater agreement will be maintained through ongoing training, and raters will rate at least 15% of the sessions to assess reliability.

Self-reported adherence measure (secondary measure of fidelity)

As a routine aspect of documentation, at every session, clinicians will complete a 5–8-item adherence checklist for “unique and essential” CPT elements that are embedded in required VA template clinical notes and which will also be embedded in the CROMIS/OQ platforms.

Adaptation

To assess adaptation (hypothesis 4), we will use a framework and coding system of modifications and adaptations made to EBPs [88] that has previously been applied to interview and observation data on CPT adaptation [26]. Session recordings will be independently rated to identify modifications.

Penetration, clinician satisfaction, LC engagement, and CPT activity reporting (P&S)

As in our consultation study, at baseline, 4, 8, and 12 months, and at the follow-up, clinicians will complete a survey on CPT activity. Questions will include number/proportion of PTSD cases who received CPT since the last assessment, number of CPT sessions delivered, satisfaction with the IS over the past month; frequency of non-study consultation or training; contextual adaptations; and confidence in CPT delivery. To assess the validity of self-reported penetration, detailed interviews with a subset of clinicians will assess eligibility of each patient with PTSD on the caseload and identify CPT elements utilized with each.

Capacity

To measure workforce capacity to deliver CPT (hypothesis 6), we will examine the proportion of clinicians in each clinic who qualify for a more advanced form of provider status, quality-rated CPT provider status (QRCPS) at each assessment. QRCPS is awarded to clinicians who have conducted CPT with at least three patients in the past year, and who achieve an average of two (out of three points) or higher adherence score and an average of four (out of six, indicating good) competence scores on a randomly selected session.
To assess organizational capacity and contextual factors (hypothesis 7), we will use two measures.
The Learning Organization Inventory (LOI) [80] is a psychometrically valid assessment of learning environment, concrete learning processes, and whether leaders reinforce learning. Given the potential for CQI to foster development of learning organizations, LOI will be assessed at two additional times for our analysis of potential mechanisms. We will also assess organization- and clinician-level constructs that have been associated with implementation success in prior research [27, 89]. The Implementation Climate Assessment (ICA) [89] will be used as an additional measure of capacity.

Potential moderators (exploratory)

Additional measures will be used to explore organization- and clinician-level moderating effects on the impact of ISs and/or use of EBPs, they will be administered at the baseline, 4-, 6-,12-, and 24-months timepoints (also to examine change over time) of the study [90].
The Organizational Social Context (OSC) measures climate, culture, and work attitudes. It is psychometrically sound [91] and predicted sustainability in prior research [27].
The Implementation Leadership Scale (ILS) [92] is a brief measure of (1) proactive, (2) knowledgeable, (3) supportive, (4) perseverant leadership with excellent internal consistency, convergent, and discriminant validity.
The Clinician Demographic Characteristics and Experience Questionnaire (CDCEQ) will be administered at baseline to assess relevant clinician demographic information and experience.
The Perceived Characteristics of Intervention Scale (PCIS) [93] is a brief survey that assesses innovation characteristics first identified by Rogers that are hypothesized to influence adoption and sustainability [93].

Interviews and qualitative strategy

We will also interview a subset of clinicians and administrators to contextualize and extend findings from the quantitative data collection. We will sample from each clinic at baseline. At post-LC and follow-up, a purposive sampling strategy will be utilized to ensure representation across systems and to capture perspectives of clinicians and administrators who experience different outcomes within each IS condition (e.g., low vs. high penetration, fidelity, and CPT utilization). The interviews will assess multilevel influences on engagement in ISs and on CPT delivery. The interview guide will be based on the tenants of the DSF [60] and the Consolidated Framework for Implementation Research [94].

Analytic strategy

We will examine descriptive statistics and distribution of all variables. Outcome variables will be transformed as needed if data violate assumptions of normality. We will evaluate effectiveness of randomization by comparing baseline measures of the key variables and demographics using χ2 and t tests.
Aim 1
The first aim is to examine the impact of two LC interventions for sustained EBP delivery on the use and effectiveness of CPT. We will test hypothesis 1A (non-inferiority of FID relative to CQI) by examining [95] whether the difference in change between the groups after the ISs is complete (12 months) and at follow-up is smaller than a predetermined clinically reliable difference (i.e., the non-inferiority margin “delta”).
To estimate change, we will use multilevel regression (i.e., growth curve analyses, mixed effect regression, hierarchical linear modeling), which offers numerous strengths for analyzing change over time for nested data, including efficiency in handling missing data, powerful and accurate estimation procedures adjusting for clustering, and modeling flexibility (e.g., allows for the inclusion of continuous or categorical, time invariant or time varying, covariates and predictors). This approach is extremely robust to missing data and considered “state of the art” for analyzing unbalanced data sets due to dropouts and loss to follow-up [96].
A three-level model will be evaluated to test the impact of IS on change in PTSD. Time (since pre-treatment) will be entered as a level-1 predictor to estimate change in PCL-5 scores. Level 2 will be included to account for the individual assessments nested within patients. Since randomization occurred at the clinic level, IS (dummy coded variable) will be entered as a level-3 predictor to examine the impact of IS on change over time in PTSD symptoms. Given the small number of systems, system will be a level-3 (clinic level) variable to assess and adjust for its influence. We will examine the influence of potentially significant covariates (e.g., baseline PCL-5, VA vs. community; veteran status, other clinician, or patient characteristics/diagnoses) and include in analyses if necessary.
To examine hypothesis 1B (that both ISs will be superior to no intervention), we will conduct piecewise multilevel growth modeling, which will allow us to estimate separate trajectories for each phase of the study by condition. This approach is being increasingly used in randomized controlled trials to more clearly elucidate change across multiple phases [97].
To further compare the impact of the ISs to no intervention (hypothesis 1B), we will use PCL and diagnostic data extracted from VAC’s CROMIS and the VA records. We will separately examine administrative PCL data for all patients with PTSD seen by VAC and VA clinicians through CROMIS and VA’s clinical CPT templates and clinical reminder data, both of which have capacity for clinic and clinician-level linkage and data extraction.
Hypothesis 1C. Different mechanisms of change will be evident for the two ISs. We hypothesize that FID will impact outcomes through increased fidelity and that CQI will impact outcomes through adaptation and development of learning organizations. Lagged regression modeling is a useful way to examine temporal precedence between contemporaneous time-varying phenomena, where a causal effect of variable X1 on X2 is established when X1 at time 1 significantly contributes to the behavior of X2 at time 2 while controlling for the effect of X2 at time 1 [98, 99]. To evaluate temporal precedence, lagged multivariate multilevel models will be evaluated [100]. Multivariate models will allow for the inclusion of two outcomes in the same model [100]. Each of the outcomes will be predicted by the level of that outcome at the previous time point (i.e., autoregressive effect, e.g., PTSD at time 2 will be predicted by PTSD at time 1) and the other variable (observer-rated fidelity, adaptation, or LOS score) in the model at the previous assessment occasion (i.e., cross-lagged path PTSD will be predicted by the proposed mechanism and vice-versa). We will also account for overall increases and decreases in the model by including time as a predictor of each outcome.
Sample size justification and power calculations
We require a sufficient sample to detect non-inferiority based on a clinically significant change of 5 points on the PCL [101, 102]. Calculations of a design effect (measure of how the design effects the standard error of the parameters) [103, 104] accounted for clustering. We computed intraclass correlations (ICC) with repeated observations of CPT patients at the clinic (ρ = 0.12), clinician (ρ = 0.16), and patient (ρ = 0.38) level in our prior research in Canada and VA, yielding a design effect of up to 2.32 [83]. Power to detect a clinically meaningful difference on PTSD symptom scores over time, taking the design effect into account, is .80 in a two-sided test, using repeated measures, with an alpha-level of 0.05 and a standard deviation (sd) of 7 points (based on prior research), for a sample size during the IS phase of 288 patients (144 per condition) within 32 clusters. In the proposed study, we expect to randomize 32 clinics, with an average of three clinicians per clinic and at least four patients per clinician during both the IS and follow-up phase. For each patient, PTSD symptoms will be measured at nine time points, as it is typical to assess symptoms frequently in CPT (baseline, every other session, post-treatment, and follow-up). To very conservatively account for potential patient-level attrition and missing data during treatment, we estimated an average of four observations per patient. Furthermore, we considered the longer study length, rates of turnover in each system, attrition rate in previous studies, balanced with participation in an active intervention, system support, and incentives, to anticipate substantial, but not unmanageable, attrition. Full participation in the IS and follow-up would yield 384 patients and at least 1536 observations, and thus, we could see a clinician attrition rate of 33% while remaining adequately powered to detect non-inferiority of CQI, and superiority of the ISs as compared to the baseline/non-IS phase data (an additional 2 patients per clinician; up to 192 patients). This projection exceeds the rate of attrition seen in our prior studies in these systems and is sufficient to test for non-inferiority on OQ measure as a secondary analysis, as the OQ-45 has an RCI of 14 (sd = 16) [105]. Finally, the sample will allow for a stable estimate of a fidelity based on the recommended G = .70–.80 level of dependability) [106].
3.I.5 Testing of additional hypotheses and exploratory analyses. To compare the budgetary impact of each IS (hypothesis 2), a budget impact analysis (BIA) [107] will be conducted to estimate the cost to the system of implementing and sustaining each IS. Results will be interpreted in conjunction with the findings for hypothesis 1A. If evidence of non-inferiority is found, the less expensive IS may be warranted, but significant differences in favor of the more costly IS might justify the added expense.
Hypotheses 3–7 involve continuous outcome data and will employ a similar multilevel regression strategy as described in section 3.I.1. We will also examine interactions between these variables and ISs to assess for a moderating effect on patient outcomes and will similarly explore potential moderating effects of attitudes (EBPAS) and social context (OSC) and leadership (ILS) on the key outcomes of interest. We will explore whether the system impacts the effects of the ISs by testing interactions between system, IS, and time.
Aim 2
The second aim is to assess each IS’s acceptability and perceived fit in diverse mental health systems.
Qualitative and mixed method analytic strategy
We will use interviews, qualitative strategies, and surveys to understand barriers and facilitators of the ISs and to CPT. Qualitative and quantitative data will be integrated to facilitate a fine-grained understanding of processes and characteristics that influence clinicians’ use and perceptions of CPT as well as fit and effectiveness of ISs within each system. A priori codes will be based on the DSF and Aarons et al.’s model of EBP implementation and sustainability [34, 85]. Using NVIVO 11 software, we will code transcripts from different stakeholders and through consensus, and identify and operationalize additional emerging codes. We will identify the central themes by diversity and triangulation across data sources, the frequency with which specific influences were identified, and attention to the key decision points and interactions described in data sources. Survey data on contextual factors will be examined in conjunction with qualitative data for the purposes of contextualization, validation, and triangulation [70, 108] to understand how contextual factors influence experience with CPT and ISs.

Discussion

Little research has been conducted to investigate different strategies to promote the sustained use of interventions. This study will contribute to a growing literature on sustainability by employing a mixed method approach to examining two different strategies across different mental health systems, offering several innovations. The application of a hybrid-type-III design [and data on the budgetary impact of each IS] [35, 47] allows the study of relative advantages of ISs and comparison of clinical outcomes in diverse care settings. Information about budget impact, time, personnel burden, perceived fit, and the key implementation outcomes will provide meaningful guidance for policy and decision-making on future implementation efforts. Penetration, fidelity, adaptation, capacity, and patient outcomes are the key implementation outcomes [69] that have not been objectively examined in prior research [46]. Patient outcomes have not been routinely assessed in mental health sustainability research [46] 3). Fidelity and adaptation have rarely been objectively assessed, and prior efforts to objectively evaluate fidelity have lacked full participation. This study is also one of the first examinations of the tenets of the Dynamic Sustainability Framework (DSF) [60], which encourages CQI and adaptation to improve sustainability and clinical outcomes. Many previous implementation studies have focused on either initial implementation, changing clinician behavior [51, 105] and skill or on promoting change at the leadership, organization, and system level [34]. Testing clinic-based CQI will allow us to examine the extent to which clinician-led, data-driven mutual adaptation can facilitate sustainability. We anticipate that this research will yield several products: a manual and toolkit for effective IS(s), papers describing implementation and patient-level outcomes, an examination of mechanisms of influence for each implementation strategy, papers reporting on multilevel predictors of sustained CPT implementation, a mixed method comparison of system, clinic, and clinician experiences of ISs in three mental health systems and comparisons of clinician vs. observer-generated data on fidelity and adaptation.

Acknowledgements

Not applicable.

Funding

This project is supported by the Canadian Institute of Health Research, funding reference number 137012 and the National Institute of Health, grant number 5R01MH106506-02.

Availability of data and materials

Data are being collected for this study. The information here is an abbreviated version of the full protocol, which can be obtained from the first author. Upon completion of the study, resources produced through this research (e.g., manuals, toolkits) will be made available.

Authors’ contributions

The study was conceptualized by SWS, CM, EF, and NS with input from JC, LD, KK, JB, DG, MS, RHS, and CG. Suggestions for refinement of study procedures and assessments were made by TM, KM, VR, and MB. All authors reviewed the manuscript for intellectual content and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.
Not applicable.
This study was approved by the institutional review board at Stanford University and VA Palo Alto Healthcare System, Palo Alto, CA; Ryerson University, Toronto, CA; University of Texas Health Sciences Center, San Antonio, TX; Boston University and VA Boston Healthcare System, Boston, MA; Suffolk University, Boston, MA; Yale University, New Haven, CT; and University of San Diego, San Diego, CA.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://​creativecommons.​org/​licenses/​by/​4.​0/​), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://​creativecommons.​org/​publicdomain/​zero/​1.​0/​) applies to the data made available in this article, unless otherwise stated.
Literatur
1.
Zurück zum Zitat McHugh RK, Barlow DH. The dissemination and implementation of evidence-based psychological treatments: a review of current efforts. Am Psychol. 2010;65:73–84.CrossRefPubMed McHugh RK, Barlow DH. The dissemination and implementation of evidence-based psychological treatments: a review of current efforts. Am Psychol. 2010;65:73–84.CrossRefPubMed
2.
Zurück zum Zitat Karlin BE, Cross G. From the laboratory to the therapy room: national dissemination and implementation of evidence-based psychotherapies in the US Department of Veterans Affairs Health Care System. Am Psychol. 2014;69(1):19–33.CrossRefPubMed Karlin BE, Cross G. From the laboratory to the therapy room: national dissemination and implementation of evidence-based psychotherapies in the US Department of Veterans Affairs Health Care System. Am Psychol. 2014;69(1):19–33.CrossRefPubMed
3.
Zurück zum Zitat Karlin BE, Brown GK, Trockel M, Cunning D, Zeiss AM, Taylor CB. National dissemination of cognitive behavioral therapy for depression in the Department of Veterans Affairs Health Care System: therapist and patient level outcomes. J Consult Clin Psychol. 2012;80(5):707.CrossRefPubMed Karlin BE, Brown GK, Trockel M, Cunning D, Zeiss AM, Taylor CB. National dissemination of cognitive behavioral therapy for depression in the Department of Veterans Affairs Health Care System: therapist and patient level outcomes. J Consult Clin Psychol. 2012;80(5):707.CrossRefPubMed
4.
Zurück zum Zitat Chard KM, Ricksecker EG, Healy ET, Karlin BE, Resick PA. Dissemination and experience with cognitive processing therapy. J Rehabil Res Dev. 2012;49(5):667–78.CrossRefPubMed Chard KM, Ricksecker EG, Healy ET, Karlin BE, Resick PA. Dissemination and experience with cognitive processing therapy. J Rehabil Res Dev. 2012;49(5):667–78.CrossRefPubMed
5.
Zurück zum Zitat Eftekhari A, Ruzek JI, Crowley JJ, Rosen CS, Greenbaum MA, Karlin BE. Effectiveness of national implementation of prolonged exposure therapy in Veterans Affairs care. JAMA Psychiatry. 2013;70(9):949–55.CrossRefPubMed Eftekhari A, Ruzek JI, Crowley JJ, Rosen CS, Greenbaum MA, Karlin BE. Effectiveness of national implementation of prolonged exposure therapy in Veterans Affairs care. JAMA Psychiatry. 2013;70(9):949–55.CrossRefPubMed
6.
Zurück zum Zitat Shiner B, D’Avolio LW, Nguyen TM, Zayed MH, Young-Xu Y, Desai RA, Schnurr PP, Fiore LD, Watts BV. Measuring use of evidence based psychotherapy for posttraumatic stress disorder. Adm Policy Ment Health Ment Health Serv Res. 2013;40(4):311–8.CrossRef Shiner B, D’Avolio LW, Nguyen TM, Zayed MH, Young-Xu Y, Desai RA, Schnurr PP, Fiore LD, Watts BV. Measuring use of evidence based psychotherapy for posttraumatic stress disorder. Adm Policy Ment Health Ment Health Serv Res. 2013;40(4):311–8.CrossRef
7.
Zurück zum Zitat Lu MW, Plagge JM, Marsiglio MC, Dobscha SK. Clinician documentation on receipt of trauma-focused evidence-based psychotherapies in a VA PTSD clinic. J Behav Health Serv Res. 2016;43(1):71–87.CrossRefPubMed Lu MW, Plagge JM, Marsiglio MC, Dobscha SK. Clinician documentation on receipt of trauma-focused evidence-based psychotherapies in a VA PTSD clinic. J Behav Health Serv Res. 2016;43(1):71–87.CrossRefPubMed
8.
Zurück zum Zitat Osei-Bonsu PE, Bolton RE, Stirman SW, Eisen SV, Herz L, Pellowe ME. Mental health providers’ decision-making around the implementation of evidence-based treatment for PTSD. J Behav Health Serv Res. 2016;7:1–1.CrossRef Osei-Bonsu PE, Bolton RE, Stirman SW, Eisen SV, Herz L, Pellowe ME. Mental health providers’ decision-making around the implementation of evidence-based treatment for PTSD. J Behav Health Serv Res. 2016;7:1–1.CrossRef
9.
Zurück zum Zitat Cook JM, Dinnen S, Simiola V, Thompson R, Schnurr PP. VA residential provider perceptions of dissuading factors to the use of two evidence-based PTSD treatments. Prof Psychol: Res Pract. 2014;45(2):136–42.CrossRef Cook JM, Dinnen S, Simiola V, Thompson R, Schnurr PP. VA residential provider perceptions of dissuading factors to the use of two evidence-based PTSD treatments. Prof Psychol: Res Pract. 2014;45(2):136–42.CrossRef
10.
Zurück zum Zitat Cook JM, Dinnen S, Simiola V, Thompson R, Schnurr PP. VA residential provider perceptions of dissuading factors to use of two evidence-based PTSD treatments. Clin Psychol. In press. Cook JM, Dinnen S, Simiola V, Thompson R, Schnurr PP. VA residential provider perceptions of dissuading factors to use of two evidence-based PTSD treatments. Clin Psychol. In press.
11.
Zurück zum Zitat Osei-Bonsu PE, Bokhour BG, Glickman ME, Rodrigues S, Mueller NM, Dell NS, Zhao S, Eisen SV, Elwy AR. The role of coping in depression treatment utilization for VA primary care patients. Patient Educ Couns. 2014;94(3):396–402.CrossRefPubMed Osei-Bonsu PE, Bokhour BG, Glickman ME, Rodrigues S, Mueller NM, Dell NS, Zhao S, Eisen SV, Elwy AR. The role of coping in depression treatment utilization for VA primary care patients. Patient Educ Couns. 2014;94(3):396–402.CrossRefPubMed
12.
Zurück zum Zitat Dusenbury L, Brannigan R, Falco M, Lake A. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. 2003;18(2):237–56.CrossRefPubMed Dusenbury L, Brannigan R, Falco M, Lake A. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. 2003;18(2):237–56.CrossRefPubMed
13.
Zurück zum Zitat Wilk JE, West JC, Duffy FF, Herrell RK, Rae DS, Hoge CW. Use of evidence-based treatment for posttraumatic stress disorder in Army behavioral healthcare. Psychiatry. 2013;76(4):336–48.CrossRefPubMed Wilk JE, West JC, Duffy FF, Herrell RK, Rae DS, Hoge CW. Use of evidence-based treatment for posttraumatic stress disorder in Army behavioral healthcare. Psychiatry. 2013;76(4):336–48.CrossRefPubMed
14.
Zurück zum Zitat Waller G, Stringer H, Meyer C. What cognitive behavioral techniques do therapists report using when delivering cognitive behavioral therapy for the eating disorders? J Consult Clin Psychol. 2012;80(1):171.CrossRefPubMed Waller G, Stringer H, Meyer C. What cognitive behavioral techniques do therapists report using when delivering cognitive behavioral therapy for the eating disorders? J Consult Clin Psychol. 2012;80(1):171.CrossRefPubMed
15.
Zurück zum Zitat Stirman SW, Calloway A, Toder K, Miller CJ, DeVito AK, Meisel SN, Xhezo R, Evans AC, Beck AT, Crits-Christoph P. Community mental health provider modifications to cognitive therapy: implications for sustainability. Psychiatr Serv. 2013;64(10):1056–9.CrossRef Stirman SW, Calloway A, Toder K, Miller CJ, DeVito AK, Meisel SN, Xhezo R, Evans AC, Beck AT, Crits-Christoph P. Community mental health provider modifications to cognitive therapy: implications for sustainability. Psychiatr Serv. 2013;64(10):1056–9.CrossRef
16.
Zurück zum Zitat Aarons GA, Miller EA, Green AE, Perrott JA, Bradway R. Adaptation happens: a qualitative case study of implementation of the incredible years evidence-based parent training program in a residential substance abuse treatment program. J Children’s Serv. 2012;7(4):233–45.CrossRef Aarons GA, Miller EA, Green AE, Perrott JA, Bradway R. Adaptation happens: a qualitative case study of implementation of the incredible years evidence-based parent training program in a residential substance abuse treatment program. J Children’s Serv. 2012;7(4):233–45.CrossRef
17.
Zurück zum Zitat Castro FG, Barrera Jr M, Martinez Jr CR. The cultural adaptation of prevention interventions: resolving tensions between fidelity and fit. Prev Sci. 2004;5(1):41–5.CrossRefPubMed Castro FG, Barrera Jr M, Martinez Jr CR. The cultural adaptation of prevention interventions: resolving tensions between fidelity and fit. Prev Sci. 2004;5(1):41–5.CrossRefPubMed
18.
Zurück zum Zitat Griner D, Smith TB. Culturally adapted mental health intervention: a meta-analytic review. Psychother Theory Res Pract Train. 2006;43(4):531–48.CrossRef Griner D, Smith TB. Culturally adapted mental health intervention: a meta-analytic review. Psychother Theory Res Pract Train. 2006;43(4):531–48.CrossRef
19.
Zurück zum Zitat Bass JK, Annan J, McIvor Murray S, Kaysen D, Griffiths S, Cetinoglu T, Wachter K, Murray LK, Bolton PA. Controlled trial of psychotherapy for Congolese survivors of sexual violence. N Engl J Med. 2013;368(23):2182–91.CrossRefPubMed Bass JK, Annan J, McIvor Murray S, Kaysen D, Griffiths S, Cetinoglu T, Wachter K, Murray LK, Bolton PA. Controlled trial of psychotherapy for Congolese survivors of sexual violence. N Engl J Med. 2013;368(23):2182–91.CrossRefPubMed
20.
Zurück zum Zitat Schulz PM, Resick PA, Huber LC, Griffin MG. The effectiveness of cognitive processing therapy for PTSD with refugees in a community setting. Cogn Behav Pract. 2006;13(4):322.CrossRef Schulz PM, Resick PA, Huber LC, Griffin MG. The effectiveness of cognitive processing therapy for PTSD with refugees in a community setting. Cogn Behav Pract. 2006;13(4):322.CrossRef
21.
Zurück zum Zitat Galovski TE, Blain LM, Mott JM, Elwood L, Houle T. Manualized therapy for PTSD: flexing the structure of cognitive processing therapy. J Consult Clin Psychol. 2012;80(6):968–81.CrossRefPubMedPubMedCentral Galovski TE, Blain LM, Mott JM, Elwood L, Houle T. Manualized therapy for PTSD: flexing the structure of cognitive processing therapy. J Consult Clin Psychol. 2012;80(6):968–81.CrossRefPubMedPubMedCentral
22.
Zurück zum Zitat Weisz JR, Chorpita BF, Palinkas LA, Schoenwald SK, Miranda J, Bearman SK, Daleiden EL, Ugueto AM, Ho A, Martin J, et al. Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: a randomized effectiveness trial. Arch Gen Psychiatry. 2012;69(3):274–82.CrossRefPubMed Weisz JR, Chorpita BF, Palinkas LA, Schoenwald SK, Miranda J, Bearman SK, Daleiden EL, Ugueto AM, Ho A, Martin J, et al. Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: a randomized effectiveness trial. Arch Gen Psychiatry. 2012;69(3):274–82.CrossRefPubMed
23.
Zurück zum Zitat Von Ranson KM, Wallace LM, Stevenson A. Psychotherapies provided for eating disorders by community clinicians: infrequent use of evidence-based treatment. Psychother Res. 2013;23(3):333–43.CrossRef Von Ranson KM, Wallace LM, Stevenson A. Psychotherapies provided for eating disorders by community clinicians: infrequent use of evidence-based treatment. Psychother Res. 2013;23(3):333–43.CrossRef
24.
Zurück zum Zitat Schulte D, Künzel R, Pepping G, Schulte-Bahrenberg T. Tailor-made versus standardized therapy of phobic patients. Advanc Behav Res Ther. 1992;14(2):67.CrossRef Schulte D, Künzel R, Pepping G, Schulte-Bahrenberg T. Tailor-made versus standardized therapy of phobic patients. Advanc Behav Res Ther. 1992;14(2):67.CrossRef
25.
Zurück zum Zitat Kennedy MG, Mizuno Y, Hoffman R, Baume C, Strand J. The effect of tailoring a model HIV prevention program for local adolescent target audiences. AIDS Educ Prev. 2000;12(3):225–38.PubMed Kennedy MG, Mizuno Y, Hoffman R, Baume C, Strand J. The effect of tailoring a model HIV prevention program for local adolescent target audiences. AIDS Educ Prev. 2000;12(3):225–38.PubMed
26.
Zurück zum Zitat Cook JM, Dinnen S, Thompson R, Simiola V, Schnurr PP. Changes in implementation of two evidence-based psychotherapies for PTSD in VA residential treatment programs: a national investigation. J Trauma Stress. 2014:n/a-n/a. Cook JM, Dinnen S, Thompson R, Simiola V, Schnurr PP. Changes in implementation of two evidence-based psychotherapies for PTSD in VA residential treatment programs: a national investigation. J Trauma Stress. 2014:n/a-n/a.
27.
Zurück zum Zitat Glisson C, Schoenwald SK, Kelleher K, Landsverk J, Hoagwood KE, Mayberg S, Green P. Therapist turnover and new program sustainability in mental health clinics as a function of organizational culture, climate, and service structure. Adm Policy Ment Health Ment Health Serv Res. 2008;35(1–2):124.CrossRef Glisson C, Schoenwald SK, Kelleher K, Landsverk J, Hoagwood KE, Mayberg S, Green P. Therapist turnover and new program sustainability in mental health clinics as a function of organizational culture, climate, and service structure. Adm Policy Ment Health Ment Health Serv Res. 2008;35(1–2):124.CrossRef
28.
Zurück zum Zitat Ehlers A, Grey N, Wild J, Stott R, Liness S, Deale A, Handley R, Albert I, Cullen D, Hackmann A. Implementation of cognitive therapy for PTSD in routine clinical care: effectiveness and moderators of outcome in a consecutive sample. Behav Res Ther. 2013;51(11):742–52.CrossRefPubMedPubMedCentral Ehlers A, Grey N, Wild J, Stott R, Liness S, Deale A, Handley R, Albert I, Cullen D, Hackmann A. Implementation of cognitive therapy for PTSD in routine clinical care: effectiveness and moderators of outcome in a consecutive sample. Behav Res Ther. 2013;51(11):742–52.CrossRefPubMedPubMedCentral
29.
Zurück zum Zitat Schoenwald SK, Garland AF, Chapman JE, Frazier SL, Sheidow AJ, Southam-Gerow MA. Toward the effective and efficient measurement of implementation fidelity. Adm Policy Ment Health. 2011;38(1):32–43.CrossRefPubMedPubMedCentral Schoenwald SK, Garland AF, Chapman JE, Frazier SL, Sheidow AJ, Southam-Gerow MA. Toward the effective and efficient measurement of implementation fidelity. Adm Policy Ment Health. 2011;38(1):32–43.CrossRefPubMedPubMedCentral
30.
Zurück zum Zitat Surís A, Link‐Malcolm J, Chard K, Ahn C, North C. A randomized clinical trial of cognitive processing therapy for Veterans with PTSD related to military sexual trauma. J Trauma Stress. 2013;26(1):28–37.CrossRefPubMed Surís A, Link‐Malcolm J, Chard K, Ahn C, North C. A randomized clinical trial of cognitive processing therapy for Veterans with PTSD related to military sexual trauma. J Trauma Stress. 2013;26(1):28–37.CrossRefPubMed
31.
Zurück zum Zitat Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89:1322–7.CrossRefPubMedPubMedCentral Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89:1322–7.CrossRefPubMedPubMedCentral
32.
Zurück zum Zitat Tanielian T, Jaycox L. Invisible wounds of war : psychological and cognitive injuries, their consequences, and services to assist recovery. Santa Monica: RAND Corporation; 2008. Tanielian T, Jaycox L. Invisible wounds of war : psychological and cognitive injuries, their consequences, and services to assist recovery. Santa Monica: RAND Corporation; 2008.
34.
Zurück zum Zitat Aarons G, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health Ment Health Serv Res. 2011;38(1):4–23.CrossRef Aarons G, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health Ment Health Serv Res. 2011;38(1):4–23.CrossRef
35.
Zurück zum Zitat Brown PM, Cameron LD, Ramondt S. Sustainability of behavioral interventions: beyond cost-effectiveness analysis. Int J Behav Med. 2015;22(3):425–33.CrossRefPubMed Brown PM, Cameron LD, Ramondt S. Sustainability of behavioral interventions: beyond cost-effectiveness analysis. Int J Behav Med. 2015;22(3):425–33.CrossRefPubMed
36.
Zurück zum Zitat Kessler RC. Posttraumatic stress disorder: the burden to the individual and to society. J Clin Psychiatry. 2000;61 suppl 5:4–12.PubMed Kessler RC. Posttraumatic stress disorder: the burden to the individual and to society. J Clin Psychiatry. 2000;61 suppl 5:4–12.PubMed
37.
Zurück zum Zitat Lunney CA, Schnurr PP. Domains of quality of life and symptoms in male Veterans treated for posttraumatic stress disorder. J Trauma Stress. 2007;20(6):955–64.CrossRefPubMed Lunney CA, Schnurr PP. Domains of quality of life and symptoms in male Veterans treated for posttraumatic stress disorder. J Trauma Stress. 2007;20(6):955–64.CrossRefPubMed
38.
Zurück zum Zitat Ahrens J, Rexford L. Cognitive processing therapy for incarcerated adolescents with PTSD. J Aggress Maltreat Trauma. 2002;6(1):201–16.CrossRef Ahrens J, Rexford L. Cognitive processing therapy for incarcerated adolescents with PTSD. J Aggress Maltreat Trauma. 2002;6(1):201–16.CrossRef
39.
Zurück zum Zitat Chard KM. An evaluation of cognitive processing therapy for the treatment of posttraumatic stress disorder related to childhood sexual abuse. J Consult Clin Psychol. 2005;73(5):965.CrossRefPubMed Chard KM. An evaluation of cognitive processing therapy for the treatment of posttraumatic stress disorder related to childhood sexual abuse. J Consult Clin Psychol. 2005;73(5):965.CrossRefPubMed
40.
Zurück zum Zitat Resick PA, Nishith P, Weaver TL, Astin MC, Feuer CA. A comparison of cognitive-processing therapy with prolonged exposure and a waiting condition for the treatment of chronic posttraumatic stress disorder in female rape victims. J Consult Clin Psychol. 2002;70(4):867.CrossRefPubMedPubMedCentral Resick PA, Nishith P, Weaver TL, Astin MC, Feuer CA. A comparison of cognitive-processing therapy with prolonged exposure and a waiting condition for the treatment of chronic posttraumatic stress disorder in female rape victims. J Consult Clin Psychol. 2002;70(4):867.CrossRefPubMedPubMedCentral
41.
Zurück zum Zitat Kaysen D, Lindgren K, Zangana GAS, Murray L, Bass J, Bolton P. Adaptation of cognitive processing therapy for treatment of torture victims: experience in Kurdistan, Iraq. Psychological Trauma: Theory, Research, Practice, and Policy. 2011;5(2):184. Kaysen D, Lindgren K, Zangana GAS, Murray L, Bass J, Bolton P. Adaptation of cognitive processing therapy for treatment of torture victims: experience in Kurdistan, Iraq. Psychological Trauma: Theory, Research, Practice, and Policy. 2011;5(2):184.
42.
Zurück zum Zitat Kaysen D, Schumm J, Pedersen ER, Seim RW, Bedard-Gilligan M, Chard K. Cognitive processing therapy for veterans with comorbid PTSD and alcohol use disorders. Addict Behav. 2014;39(2):420–7.CrossRefPubMed Kaysen D, Schumm J, Pedersen ER, Seim RW, Bedard-Gilligan M, Chard K. Cognitive processing therapy for veterans with comorbid PTSD and alcohol use disorders. Addict Behav. 2014;39(2):420–7.CrossRefPubMed
43.
Zurück zum Zitat Beidas R, Kendall PC. Training therapists in evidence-based practice: a critical review of studies from a systems-contextual perspective. Clin Psychol Sci Pract. 2010;17(1):1.CrossRef Beidas R, Kendall PC. Training therapists in evidence-based practice: a critical review of studies from a systems-contextual perspective. Clin Psychol Sci Pract. 2010;17(1):1.CrossRef
44.
Zurück zum Zitat Herschell AD, Kolko DJ, Baumann BL, Davis AC. The role of therapist training in the implementation of psychosocial treatments: a review and critique with recommendations. Clin Psychol Rev. 2010;30(4):448–66.CrossRefPubMedPubMedCentral Herschell AD, Kolko DJ, Baumann BL, Davis AC. The role of therapist training in the implementation of psychosocial treatments: a review and critique with recommendations. Clin Psychol Rev. 2010;30(4):448–66.CrossRefPubMedPubMedCentral
45.
Zurück zum Zitat Rakovshik SG, McManus F. Establishing evidence-based training in cognitive behavioral therapy: a review of current empirical findings and theoretical guidance. Clin Psychol Rev. 2010;30(5):496–516.CrossRefPubMed Rakovshik SG, McManus F. Establishing evidence-based training in cognitive behavioral therapy: a review of current empirical findings and theoretical guidance. Clin Psychol Rev. 2010;30(5):496–516.CrossRefPubMed
46.
Zurück zum Zitat Stirman SW, Kimberly JR, Calloway A, Cook N, Castro F, Charns MP. The sustainability of new programs and interventions: a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7:17.CrossRef Stirman SW, Kimberly JR, Calloway A, Cook N, Castro F, Charns MP. The sustainability of new programs and interventions: a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7:17.CrossRef
47.
Zurück zum Zitat Bein E, Anderson T, Strupp HH, Henry WP, Schacht TE, Binder J, Butler S. The effects of training in time-limited dynamic psychotherapy: changes in therapeutic outcome. Psychother Res. 2000;10(2):119–32.CrossRefPubMed Bein E, Anderson T, Strupp HH, Henry WP, Schacht TE, Binder J, Butler S. The effects of training in time-limited dynamic psychotherapy: changes in therapeutic outcome. Psychother Res. 2000;10(2):119–32.CrossRefPubMed
48.
Zurück zum Zitat Fauth J, Mathisen A, Smith S. Reflections on a time-limited dynamic psychotherapy training project: new directions for brief psychotherapy training. In: North American Society of Psychotherapy Research 2006 Annual Conference 2006; Columbus, OH. 2006. Fauth J, Mathisen A, Smith S. Reflections on a time-limited dynamic psychotherapy training project: new directions for brief psychotherapy training. In: North American Society of Psychotherapy Research 2006 Annual Conference 2006; Columbus, OH. 2006.
49.
Zurück zum Zitat Webb CA, Derubeis RJ, Barber JP. Therapist adherence/competence and treatment outcome: a meta-analytic review. J Consult Clin Psychol. 2010;78(2):200–11.CrossRefPubMedPubMedCentral Webb CA, Derubeis RJ, Barber JP. Therapist adherence/competence and treatment outcome: a meta-analytic review. J Consult Clin Psychol. 2010;78(2):200–11.CrossRefPubMedPubMedCentral
50.
Zurück zum Zitat Farmer CC, Mitchell KS, Parker-Guilbert K, Galovski TE. Fidelity to the cognitive processing therapy protocol: evaluation of critical elements. Behav Ther. 2016. Farmer CC, Mitchell KS, Parker-Guilbert K, Galovski TE. Fidelity to the cognitive processing therapy protocol: evaluation of critical elements. Behav Ther. 2016.
51.
Zurück zum Zitat Schoenwald SK, Sheidow AJ, Chapman JE. Clinical supervision in treatment transport: effects on adherence and outcomes. J Consult Clin Psychol. 2009;77(3):410–21.CrossRefPubMedPubMedCentral Schoenwald SK, Sheidow AJ, Chapman JE. Clinical supervision in treatment transport: effects on adherence and outcomes. J Consult Clin Psychol. 2009;77(3):410–21.CrossRefPubMedPubMedCentral
52.
Zurück zum Zitat Watts BV, Shiner B, Zubkoff L, Carpenter-Song E, Ronconi JM, Coldwell CM. Implementation of evidence-based psychotherapies for posttraumatic stress disorder in VA specialty clinics. Psychiatr Serv. 2014;65(5):648–53.CrossRefPubMed Watts BV, Shiner B, Zubkoff L, Carpenter-Song E, Ronconi JM, Coldwell CM. Implementation of evidence-based psychotherapies for posttraumatic stress disorder in VA specialty clinics. Psychiatr Serv. 2014;65(5):648–53.CrossRefPubMed
53.
Zurück zum Zitat Aarons GA, Sommerfeld DH, Hecht DB, Silovsky JF, Chaffin MJ. The impact of evidence-based practice implementation and fidelity monitoring on staff turnover: evidence for a protective effect. J Consult Clin Psychol. 2009;77(2):270.CrossRefPubMedPubMedCentral Aarons GA, Sommerfeld DH, Hecht DB, Silovsky JF, Chaffin MJ. The impact of evidence-based practice implementation and fidelity monitoring on staff turnover: evidence for a protective effect. J Consult Clin Psychol. 2009;77(2):270.CrossRefPubMedPubMedCentral
54.
Zurück zum Zitat Harned MS, Dimeff LA, Woodcock EA, Contreras I. Predicting adoption of exposure therapy in a randomized controlled dissemination trial. J Anxiety Disord. 2013;27(8):754–62.CrossRefPubMed Harned MS, Dimeff LA, Woodcock EA, Contreras I. Predicting adoption of exposure therapy in a randomized controlled dissemination trial. J Anxiety Disord. 2013;27(8):754–62.CrossRefPubMed
55.
Zurück zum Zitat Bond GR, Drake RE, McHugo GJ, Peterson AE, Jones AM, Williams J. Long-term sustainability of evidence-based practices in community mental health agencies. Adm Policy Ment Health Ment Health Serv Res. 2014;41(2):228–36.CrossRef Bond GR, Drake RE, McHugo GJ, Peterson AE, Jones AM, Williams J. Long-term sustainability of evidence-based practices in community mental health agencies. Adm Policy Ment Health Ment Health Serv Res. 2014;41(2):228–36.CrossRef
56.
Zurück zum Zitat Creed TA, Stirman SW, Evans AC, Beck AT. A model for implementation of cognitive therapy in community mental health: The Beck Initiative. Behav Therapist. 2014;37(3):56-64. Creed TA, Stirman SW, Evans AC, Beck AT. A model for implementation of cognitive therapy in community mental health: The Beck Initiative. Behav Therapist. 2014;37(3):56-64.
57.
Zurück zum Zitat Finley EP, Garcia HA, Ketchum NS, McGeary DD, McGeary CA, Stirman SW, Peterson AL. Utilization of evidence-based psychotherapies in Veterans Affairs posttraumatic stress disorder outpatient clinics. Psychological services. 2015;12(1):73.CrossRefPubMed Finley EP, Garcia HA, Ketchum NS, McGeary DD, McGeary CA, Stirman SW, Peterson AL. Utilization of evidence-based psychotherapies in Veterans Affairs posttraumatic stress disorder outpatient clinics. Psychological services. 2015;12(1):73.CrossRefPubMed
58.
Zurück zum Zitat Stirman SW, Gutierrez-Colina A, Toder K, Castro F, Esposito G, Barg F, Beck AT, Crits-Christoph P. Clinicians’ perspectives on cognitive therapy in community mental health settings: implications for training and implementation. Adm Policy Ment Health Ment Health Serv Res. 2013;40(4):274–85.CrossRef Stirman SW, Gutierrez-Colina A, Toder K, Castro F, Esposito G, Barg F, Beck AT, Crits-Christoph P. Clinicians’ perspectives on cognitive therapy in community mental health settings: implications for training and implementation. Adm Policy Ment Health Ment Health Serv Res. 2013;40(4):274–85.CrossRef
59.
Zurück zum Zitat Stirman SW, Shields N, Deloriea J, Landy MSH, Belus JM, Maslej MM, Monson CM. A randomized controlled dismantling trial of post-workshop consultation strategies to increase effectiveness and fidelity to an evidence-based psychotherapy for posttraumatic stress disorder. Implement Sci. 2013;8(82):1–8. Stirman SW, Shields N, Deloriea J, Landy MSH, Belus JM, Maslej MM, Monson CM. A randomized controlled dismantling trial of post-workshop consultation strategies to increase effectiveness and fidelity to an evidence-based psychotherapy for posttraumatic stress disorder. Implement Sci. 2013;8(82):1–8.
60.
Zurück zum Zitat Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainability amid ongoing change. Implement Sci. 2013;8(1):117.CrossRefPubMedPubMedCentral Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainability amid ongoing change. Implement Sci. 2013;8(1):117.CrossRefPubMedPubMedCentral
61.
Zurück zum Zitat Haine-Schlagel R, Brookman-Frazee L, Janis B, Gordon J. Evaluating a learning collaborative to implement evidence-informed engagement strategies in community-based services for young children. Child Youth Care Forum. 2013;42(5):457–73.CrossRef Haine-Schlagel R, Brookman-Frazee L, Janis B, Gordon J. Evaluating a learning collaborative to implement evidence-informed engagement strategies in community-based services for young children. Child Youth Care Forum. 2013;42(5):457–73.CrossRef
62.
Zurück zum Zitat Chovil N. One small step at a time: implementing continuous quality improvement in child and youth mental health services. Child & Youth Services. 2010;31(1–2):21–34.CrossRef Chovil N. One small step at a time: implementing continuous quality improvement in child and youth mental health services. Child & Youth Services. 2010;31(1–2):21–34.CrossRef
63.
Zurück zum Zitat Aarons GA, Green A, Palinkas LA, Self-Brown S, Whitaker DJ, Lutzker JR, Silovsky JF, Hecht DB, Chaffin MJ. Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implementation Sci. 2012;7(1):32.CrossRef Aarons GA, Green A, Palinkas LA, Self-Brown S, Whitaker DJ, Lutzker JR, Silovsky JF, Hecht DB, Chaffin MJ. Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implementation Sci. 2012;7(1):32.CrossRef
64.
Zurück zum Zitat LeBrasseur R, Whissell R, Ojha A. Organisational learning, transformational leadership and implementation of continuous quality improvement in Canadian hospitals. Aust J Manag. 2002;27(2):141–62.CrossRef LeBrasseur R, Whissell R, Ojha A. Organisational learning, transformational leadership and implementation of continuous quality improvement in Canadian hospitals. Aust J Manag. 2002;27(2):141–62.CrossRef
65.
Zurück zum Zitat Nadeem E, Olin SS, Hill LC, Hoagwood KE, Horwitz SM. A literature review of learning collaboratives in mental health care: used but untested. Psychiatr Serv. 2014;65(9):1088–99.CrossRefPubMedPubMedCentral Nadeem E, Olin SS, Hill LC, Hoagwood KE, Horwitz SM. A literature review of learning collaboratives in mental health care: used but untested. Psychiatr Serv. 2014;65(9):1088–99.CrossRefPubMedPubMedCentral
66.
Zurück zum Zitat Raghavan R: The role of economic evaluation in dissemination and implementation research. In: Brownson RC, Colditz G, Proctor E, editors. Dissemination and implementation research in health: translating science into practice. edn. Oxford University Press; 2012: 94–113. Raghavan R: The role of economic evaluation in dissemination and implementation research. In: Brownson RC, Colditz G, Proctor E, editors. Dissemination and implementation research in health: translating science into practice. edn. Oxford University Press; 2012: 94–113.
67.
Zurück zum Zitat Raghavan R, Bright C, Shadoin A. Toward a policy ecology of implementation of evidence-based practices in public mental health settings. Implement Sci. 2008;3(1):26.CrossRefPubMedPubMedCentral Raghavan R, Bright C, Shadoin A. Toward a policy ecology of implementation of evidence-based practices in public mental health settings. Implement Sci. 2008;3(1):26.CrossRefPubMedPubMedCentral
68.
Zurück zum Zitat Curran G, Bauer MS, Mittman BS, Pyne JM, Stetler CB. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012. Curran G, Bauer MS, Mittman BS, Pyne JM, Stetler CB. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012.
69.
Zurück zum Zitat Proctor E, Silmere H, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health Ment Health Serv Res. 2011;38(2):65-76. Proctor E, Silmere H, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health Ment Health Serv Res. 2011;38(2):65-76.
70.
Zurück zum Zitat Spillane J, Pareja A, Dorner L, Barnes C, May H, Huff J, Camburn E. Mixing methods in randomized controlled trials (RCTs): validation, contextualization, triangulation, and control. Educ Assess, Eval Account. 2010;22(1):5–28.CrossRef Spillane J, Pareja A, Dorner L, Barnes C, May H, Huff J, Camburn E. Mixing methods in randomized controlled trials (RCTs): validation, contextualization, triangulation, and control. Educ Assess, Eval Account. 2010;22(1):5–28.CrossRef
72.
Zurück zum Zitat Brown CH, Ten Have TR, Jo B, Dagne G, Wyman PA, Muthén B, Gibbons RD. Adaptive designs for randomized trials in public health. Annu Rev Public Health. 2009;30:1.CrossRefPubMedPubMedCentral Brown CH, Ten Have TR, Jo B, Dagne G, Wyman PA, Muthén B, Gibbons RD. Adaptive designs for randomized trials in public health. Annu Rev Public Health. 2009;30:1.CrossRefPubMedPubMedCentral
73.
Zurück zum Zitat Kanter JW, Tsai M, Holman G, Koerner K. Preliminary data from a randomized pilot study of web-based functional analytic psychotherapy therapist training. Psychotherapy. 2013;50(2):248.CrossRefPubMed Kanter JW, Tsai M, Holman G, Koerner K. Preliminary data from a randomized pilot study of web-based functional analytic psychotherapy therapist training. Psychotherapy. 2013;50(2):248.CrossRefPubMed
74.
Zurück zum Zitat Puspitasari A, Kanter JW, Murphy J, Crowe A, Koerner K. Developing an online, modular, active learning training program for behavioral activation. Psychotherapy. 2013;50(2):256.CrossRefPubMed Puspitasari A, Kanter JW, Murphy J, Crowe A, Koerner K. Developing an online, modular, active learning training program for behavioral activation. Psychotherapy. 2013;50(2):256.CrossRefPubMed
75.
Zurück zum Zitat Nadeem E, Olin SS, Hill LC, Hoagwood KE, Horwitz SM. Understanding the components of quality improvement collaboratives: a systematic literature review. Milbank Q. 2013;91(2):354–94.CrossRefPubMedPubMedCentral Nadeem E, Olin SS, Hill LC, Hoagwood KE, Horwitz SM. Understanding the components of quality improvement collaboratives: a systematic literature review. Milbank Q. 2013;91(2):354–94.CrossRefPubMedPubMedCentral
76.
Zurück zum Zitat Stirman SW, Shields N, Deloriea J, Landy MS, Belus JM, Maslej MM, Lane J, Monson CM. A randomized comparison of three post-workshop consultation strategies. In: Association for Behavioral and Cognitive Therapies: 2014; Philadelphia, PA. 2014. Stirman SW, Shields N, Deloriea J, Landy MS, Belus JM, Maslej MM, Lane J, Monson CM. A randomized comparison of three post-workshop consultation strategies. In: Association for Behavioral and Cognitive Therapies: 2014; Philadelphia, PA. 2014.
77.
Zurück zum Zitat Institute for Healthcare Improvement. The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement. IHI Innovation Series white paper. Boston: Institute for Healthcare Improvement; 2003. Institute for Healthcare Improvement. The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement. IHI Innovation Series white paper. Boston: Institute for Healthcare Improvement; 2003.
78.
Zurück zum Zitat Gustafson DH, Quanbeck AR, Robinson JM, Ford JH, Pulvermacher A, French MT, McConnell KJ, Batalden PB, Hoffman KA, McCarty D. Which elements of improvement collaboratives are most effective? A cluster‐randomized trial. Addiction. 2013;108(6):1145–57.CrossRefPubMedPubMedCentral Gustafson DH, Quanbeck AR, Robinson JM, Ford JH, Pulvermacher A, French MT, McConnell KJ, Batalden PB, Hoffman KA, McCarty D. Which elements of improvement collaboratives are most effective? A cluster‐randomized trial. Addiction. 2013;108(6):1145–57.CrossRefPubMedPubMedCentral
79.
Zurück zum Zitat Stirman SW, Calloway A, Rasmusson AM, McDonald A, Monson CM, Resick PA. The relationship between treatment fidelity and clinical outcomes in the implementation of CPT in VA settings. In: Association for Behavioral and Cognitive Therapies: 2011; Toronto, Canada. 2011. Stirman SW, Calloway A, Rasmusson AM, McDonald A, Monson CM, Resick PA. The relationship between treatment fidelity and clinical outcomes in the implementation of CPT in VA settings. In: Association for Behavioral and Cognitive Therapies: 2011; Toronto, Canada. 2011.
80.
Zurück zum Zitat Garvin DA, Edmondson AE, Gino F. Is yours a learning organization? Harvard Business Review. 2008;86(3):109. Garvin DA, Edmondson AE, Gino F. Is yours a learning organization? Harvard Business Review. 2008;86(3):109.
82.
Zurück zum Zitat Lambert MJ, Finch AE. The outcome questionnaire. 1999. Lambert MJ, Finch AE. The outcome questionnaire. 1999.
83.
Zurück zum Zitat Lambert MJ. Administration and scoring manual for the OQ-45.2 (Outcome Questionnaire). OQ Measures, LLC; 2004. Lambert MJ. Administration and scoring manual for the OQ-45.2 (Outcome Questionnaire). OQ Measures, LLC; 2004.
85.
Zurück zum Zitat Larsen DL, Attkisson CC, Hargreaves WA, Nguyen TD. Assessment of client/patient satisfaction: development of a general scale. Eval Program Plann. 1979;2(3):197–207.CrossRefPubMed Larsen DL, Attkisson CC, Hargreaves WA, Nguyen TD. Assessment of client/patient satisfaction: development of a general scale. Eval Program Plann. 1979;2(3):197–207.CrossRefPubMed
86.
Zurück zum Zitat Mauskopf JA, Sullivan SD, Annemans L, Caro J, Mullins CD, Nuijten M, Orlewska E, Watkins J, Trueman P. Principles of good practice for budget impact analysis: report of the ISPOR task force on good research practices—budget impact analysis. Value Health. 2007;10(5):336–47.CrossRefPubMed Mauskopf JA, Sullivan SD, Annemans L, Caro J, Mullins CD, Nuijten M, Orlewska E, Watkins J, Trueman P. Principles of good practice for budget impact analysis: report of the ISPOR task force on good research practices—budget impact analysis. Value Health. 2007;10(5):336–47.CrossRefPubMed
87.
Zurück zum Zitat Vallis TM, Shaw BF, Dobson K. The cognitive therapy scale: psychometric properties. J Consult Clin Psychol. 1986;54:381–5.CrossRefPubMed Vallis TM, Shaw BF, Dobson K. The cognitive therapy scale: psychometric properties. J Consult Clin Psychol. 1986;54:381–5.CrossRefPubMed
88.
Zurück zum Zitat Stirman SW, Miller CJ, Toder K, Calloway A. Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implementation Sci. 2013;8(65). Stirman SW, Miller CJ, Toder K, Calloway A. Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implementation Sci. 2013;8(65).
89.
Zurück zum Zitat Ehrhart MG, Aarons GA, Farahnak LR. Assessing the organizational context for EBP implementation: the development and validity testing of the Implementation Climate Scale (ICS). Implementation Science. 2014;9(1):157.CrossRefPubMedPubMedCentral Ehrhart MG, Aarons GA, Farahnak LR. Assessing the organizational context for EBP implementation: the development and validity testing of the Implementation Climate Scale (ICS). Implementation Science. 2014;9(1):157.CrossRefPubMedPubMedCentral
90.
Zurück zum Zitat Beidas R, Marcus S, Aarons GA, Hoagwood KE, Schoenwald SK, Evans AC, Hurford MO, Hadley T, Barg FK, Walsh LM, et al. Predictors of community therapists’ use of therapy techniques in a large public mental health system. JAMA Pediatr. 2015;169(4):374–82.CrossRefPubMedPubMedCentral Beidas R, Marcus S, Aarons GA, Hoagwood KE, Schoenwald SK, Evans AC, Hurford MO, Hadley T, Barg FK, Walsh LM, et al. Predictors of community therapists’ use of therapy techniques in a large public mental health system. JAMA Pediatr. 2015;169(4):374–82.CrossRefPubMedPubMedCentral
91.
Zurück zum Zitat Glisson C, Landsverk J, Schoenwald S, Kelleher K, Hoagwood K, Mayberg S, Green P. Assessing the organizational social context (OSC) of mental health services: implications for research and practice. Adm Policy Ment Health Ment Health Serv Res. 2008;35(1):98–113.CrossRef Glisson C, Landsverk J, Schoenwald S, Kelleher K, Hoagwood K, Mayberg S, Green P. Assessing the organizational social context (OSC) of mental health services: implications for research and practice. Adm Policy Ment Health Ment Health Serv Res. 2008;35(1):98–113.CrossRef
92.
Zurück zum Zitat Aarons G, Ehrhart M, Farahnak L. The implementation leadership scale (ILS): development of a brief measure of unit level implementation leadership. Implement Sci. 2014;9(1):45.CrossRefPubMedPubMedCentral Aarons G, Ehrhart M, Farahnak L. The implementation leadership scale (ILS): development of a brief measure of unit level implementation leadership. Implement Sci. 2014;9(1):45.CrossRefPubMedPubMedCentral
93.
Zurück zum Zitat Cook JM, Thompson R, Schnurr PP. Perceived Characteristics of Intervention Scale: Development and Psychometric Properties. Assessment. 2015;22(6):704–14.CrossRefPubMed Cook JM, Thompson R, Schnurr PP. Perceived Characteristics of Intervention Scale: Development and Psychometric Properties. Assessment. 2015;22(6):704–14.CrossRefPubMed
94.
Zurück zum Zitat Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50.CrossRefPubMedPubMedCentral Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50.CrossRefPubMedPubMedCentral
95.
Zurück zum Zitat Blackwelder W. Current issues in clinical equivalence trials. J Dent Res. 2004;83 suppl 1:C113–5.CrossRefPubMed Blackwelder W. Current issues in clinical equivalence trials. J Dent Res. 2004;83 suppl 1:C113–5.CrossRefPubMed
96.
Zurück zum Zitat Schafer JL, Graham JW. Missing data: our view of the state of the art. Psychol Methods. 2002;7(2):147–77.CrossRefPubMed Schafer JL, Graham JW. Missing data: our view of the state of the art. Psychol Methods. 2002;7(2):147–77.CrossRefPubMed
97.
Zurück zum Zitat Foa EB, Yusko DA, McLean CP, et al. Concurrent naltrexone and prolonged exposure therapy for patients with comorbid alcohol dependence and PTSD: a randomized clinical trial. J Am Med Assoc. 2013;310(5):488–95.CrossRef Foa EB, Yusko DA, McLean CP, et al. Concurrent naltrexone and prolonged exposure therapy for patients with comorbid alcohol dependence and PTSD: a randomized clinical trial. J Am Med Assoc. 2013;310(5):488–95.CrossRef
98.
Zurück zum Zitat Zalta AK, Gillihan SJ, Fisher AJ, Mintz J, McLean CP, Yehuda R, Foa EB. Change in negative cognitions associated with PTSD predicts symptom reduction in prolonged exposure. J Consult Clin Psychol. 2014;82(1):171.CrossRefPubMed Zalta AK, Gillihan SJ, Fisher AJ, Mintz J, McLean CP, Yehuda R, Foa EB. Change in negative cognitions associated with PTSD predicts symptom reduction in prolonged exposure. J Consult Clin Psychol. 2014;82(1):171.CrossRefPubMed
99.
Zurück zum Zitat Granger CW. Investigating causal relations by econometric models and cross-spectral methods. Econometrica: J Econometric Soc. 1969;37(3):424-38. Granger CW. Investigating causal relations by econometric models and cross-spectral methods. Econometrica: J Econometric Soc. 1969;37(3):424-38.
100.
Zurück zum Zitat Baldwin SA, Imel ZE, Braithwaite SR, Atkins DC. Analyzing multiple outcomes in clinical research using multivariate multilevel models. J Consult Clin Psychol. 2014;82(5):920.CrossRefPubMedPubMedCentral Baldwin SA, Imel ZE, Braithwaite SR, Atkins DC. Analyzing multiple outcomes in clinical research using multivariate multilevel models. J Consult Clin Psychol. 2014;82(5):920.CrossRefPubMedPubMedCentral
101.
Zurück zum Zitat Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale: Erlbaum; 1988. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale: Erlbaum; 1988.
102.
Zurück zum Zitat Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74.CrossRefPubMed Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74.CrossRefPubMed
103.
Zurück zum Zitat Raudenbush SW. Statistical analysis and optimal design for cluster randomized trials. Psychol Methods. 1997;2(2):173.CrossRef Raudenbush SW. Statistical analysis and optimal design for cluster randomized trials. Psychol Methods. 1997;2(2):173.CrossRef
104.
Zurück zum Zitat Snijders TA. Power and sample size in multilevel linear models. Encyclopedia of Statistics in Behavioral Science. 2005;3:1570–73. Snijders TA. Power and sample size in multilevel linear models. Encyclopedia of Statistics in Behavioral Science. 2005;3:1570–73.
105.
Zurück zum Zitat Nadeem E, Gleacher A, Pimentel S, Hill LC, McHugh M, Hoagwood KE. The role of consultation calls for clinic supervisors in supporting large-scale dissemination of evidence-based treatments for children. Administration and Policy in Mental Health and Mental Health Services Research. 2013;40(6):530-40. Nadeem E, Gleacher A, Pimentel S, Hill LC, McHugh M, Hoagwood KE. The role of consultation calls for clinic supervisors in supporting large-scale dissemination of evidence-based treatments for children. Administration and Policy in Mental Health and Mental Health Services Research. 2013;40(6):530-40.
106.
Zurück zum Zitat Dennhag I, Gibbons MBC, Barber JP, Gallop R, Crits-Christoph P. How many treatment sessions and patients are needed to create a stable score of adherence and competence in the treatment of cocaine dependence? Psychother Res. 2012;22(4):475–88.CrossRefPubMedPubMedCentral Dennhag I, Gibbons MBC, Barber JP, Gallop R, Crits-Christoph P. How many treatment sessions and patients are needed to create a stable score of adherence and competence in the treatment of cocaine dependence? Psychother Res. 2012;22(4):475–88.CrossRefPubMedPubMedCentral
107.
Zurück zum Zitat Garattini L, van de Vooren K. Budget impact analysis in economic evaluation: a proposal for a clearer definition. Eur J Health Econ. 2011;12(6):499–502.CrossRefPubMed Garattini L, van de Vooren K. Budget impact analysis in economic evaluation: a proposal for a clearer definition. Eur J Health Econ. 2011;12(6):499–502.CrossRefPubMed
108.
Zurück zum Zitat Palinkas L, Aarons G, Horwitz S, Chamberlain P, Hurlburt M, Landsverk J. Mixed method designs in implementation Research. Adm Policy Ment Health Ment Health Serv Res. 2010;38(1):44–53.CrossRef Palinkas L, Aarons G, Horwitz S, Chamberlain P, Hurlburt M, Landsverk J. Mixed method designs in implementation Research. Adm Policy Ment Health Ment Health Serv Res. 2010;38(1):44–53.CrossRef
Metadaten
Titel
Improving and sustaining delivery of CPT for PTSD in mental health systems: a cluster randomized trial
verfasst von
Shannon Wiltsey Stirman
Erin P. Finley
Norman Shields
Joan Cook
Rachel Haine-Schlagel
James F. Burgess Jr
Linda Dimeff
Kelly Koerner
Michael Suvak
Cassidy A. Gutner
David Gagnon
Tasoula Masina
Matthew Beristianos
Kera Mallard
Vanessa Ramirez
Candice Monson
Publikationsdatum
01.12.2017
Verlag
BioMed Central
Erschienen in
Implementation Science / Ausgabe 1/2017
Elektronische ISSN: 1748-5908
DOI
https://doi.org/10.1186/s13012-017-0544-5

Weitere Artikel der Ausgabe 1/2017

Implementation Science 1/2017 Zur Ausgabe