nach oben

Administration and Policy in Mental Health and Mental Health Services Research

Erschienen in:

21.09.2021 | Original Article

Role-Play Assessment of Therapist Adherence and Skill in Implementation of Trauma-Focused Cognitive-Behavioral Therapy

verfasst von: Brigid R. Marriott, Evelyn Cho, Siena K. Tugendrajch, Matthew D. Kliethermes, J. Curtis McMillen, Enola K. Proctor, Kristin M. Hawley

Erschienen in: Administration and Policy in Mental Health and Mental Health Services Research | Ausgabe 3/2022

Einloggen, um Zugang zu erhalten

Abstract

Numerous efforts are underway to train clinicians in evidence-based practices. Unfortunately, the field has few practical measures of therapist adherence and skill with which to judge the success of these training and implementation efforts. One possible assessment method is using behavioral rehearsal, or role-play, as an analogue for therapist in-session behavior. The current study describes aspects of reliability, validity and utility of a behavioral role-play assessment developed to evaluate therapist adherence and skill in implementing Trauma-Focused Cognitive-Behavioral Therapy (TF-CBT). TF-CBT role-play assessments were conducted with a sample of 43 therapists as part of a larger training study. The TF-CBT role-play assessments were independently coded for TF-CBT adherence and skill by a certified TF-CBT trainer and three clinical psychology doctoral students. Findings indicated good interrater reliability for the individual items (ICC: M = .71, SD = .15). Regarding utility, 67.19% (n = 43/64) of contacted therapists completed the role-play assessment, which took an average of 30 min (M = 31.42, SD = 5.65) to complete and 60 min (M = 62.84, SD = 11.31) to code. Therapists with a master’s degree were more likely to complete the role-play assessment than those with other degrees but no other differences in demographic variables, practice characteristics, or TF-CBT knowledge or training were found between participants and nonparticipants. Role-play assessments may offer an alternative to observational coding for assessing therapist adherence and skill, particularly in contexts where session recordings are not feasible.

Bearman, S. K., Schneiderman, R. L., & Zoloth, E. (2017). Building an evidence base for effective supervision practices: An analogue experiment of supervision to increase EBT fidelity. Administration and Policy in Mental Health and Mental Health Services Research, 44(2), 293–307. https://doi.org/10.1007/s10488-016-0723-8CrossRefPubMed

Beidas, R. S., Cross, W., & Dorsey, S. (2014). Show me, don’t tell me: Behavioral rehearsal as a training and analogue fidelity tool. Cognitive and Behavioral Practice, 21(1), 1–11. https://doi.org/10.1016/j.cbpra.2013.04.002CrossRefPubMed

Beidas, R. S., Edmunds, J. M., Marcus, S. C., & Kendall, P. C. (2012). Training and consultation to promote implementation of an empirically supported treatment: A randomized trial. Psychiatric Services, 63(7), 660–665. https://doi.org/10.1176/appi.ps.201100401CrossRefPubMed

Beidas, R. S., Maclean, J. C., Fishman, J., Dorsey, S., Schoenwald, S. K., Mandell, D. S., Shea, J. A., McLeod, B. D., French, M. T., Hogue, A., Adams, D. R., Lieberman, A., Becker-Haimes, E. M., & Marcus, S. C. (2016). A randomized trial to identify accurate and cost-effective fidelity measurement methods for cognitive-behavioral therapy: Project FACTS study protocol. BMC Psychiatry, 16(1), 323. https://doi.org/10.1186/s12888-016-1034-zCrossRefPubMedPubMedCentral

Beidas, R. S., Marcus, S., Aarons, G. A., Hoagwood, K. E., Schoenwald, S., Evans, A. C., Hurford, M. O., Hadley, T., Barg, F. K., Walsh, L. M., Adams, D. R., & Mandell, D. S. (2015). Predictors of community therapists’ use of therapy techniques in a large public mental health system. JAMA Pediatrics, 169(4), 374–382. https://doi.org/10.1001/jamapediatrics.2014.3736CrossRefPubMedPubMedCentral

Borntrager, C., Chorpita, B. F., Higa-McMillan, C. K., Daleiden, E. L., & Starace, N. (2013). Usual care for trauma-exposed youth: Are clinician-reported therapy techniques evidence-based? Children and Youth Services Review, 35(1), 133–141. https://doi.org/10.1016/J.CHILDYOUTH.2012.09.018CrossRef

Carroll, K. M., Nich, C., Sifry, R. L., Nuro, K. F., Frankforter, T. L., Ball, S. A., Fenton, L., & Rounsaville, B. J. (2000). A general system for evaluating therapist adherence and competence in psychotherapy research in the addictions. Drug and Alcohol Dependence, 57(3), 225–238. https://doi.org/10.1016/S0376-8716(99)00049-6CrossRefPubMed

Chapman, J. E., Mccart, M. R., Letourneau, E. J., & Sheidow, A. J. (2013). Comparison of youth, caregiver, therapist, trained, and treatment expert raters of therapist adherence to a substance abuse treatment protocol. Journal of Consulting and Clinical Psychology, 81(4), 674–680. https://doi.org/10.1037/a0033021CrossRefPubMedPubMedCentral

Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290. https://doi.org/10.1037/1040-3590.6.4.284CrossRef

Cohen, J. A., Mannarino, A. P., & Deblinger, E. (2006). Treating trauma and traumatic grief in children and adolescents. Guilford Press.

Collyer, H., Eisler, I., & Woolgar, M. (2019). Systematic literature review and meta-analysis of the relationship between adherence, competence and outcome in psychotherapy for children and adolescents. European Child and Adolescent Psychiatry. https://doi.org/10.1007/s00787-018-1265-2CrossRefPubMed

Cooper, Z., Doll, H., Bailey-Straebler, S., Bohn, K., de Vries, D., Murphy, R., O’Connor, M. E., & Fairburn, C. G. (2017). Assessing therapist competence: development of a performance-based measure and its comparison with a web-based measure. JMIR Mental Health, 4(4), e51. https://doi.org/10.2196/mental.7704CrossRefPubMedPubMedCentral

Cross, W. F., Seaburn, D., Gibbs, D., Schmeelk-Cone, K., White, A. M., & Caine, E. D. (2011). Does practice make perfect? A randomized control trial of behavioral rehearsal on suicide prevention gatekeeper skills. The Journal of Primary Prevention, 32(3–4), 195–211. https://doi.org/10.1007/s10935-011-0250-zCrossRefPubMedPubMedCentral

Dillman, D. (2000). Procedures for conducting government-sponsored establishment surveys: Comparisons of the total design method (TDM), a traditional cost-compensation. Proceedings of American Statistical Association, Second International Conference on Establishment Surveys (pp. 343–352).

Dimeff, L. A., Koerner, K., Woodcock, E. A., Beadnell, B., Brown, M. Z., Skutch, J. M., Paves, A. P., Bazinet, A., & Harned, M. S. (2009). Which training method works best? A randomized controlled trial comparing three methods of training clinicians in dialectical behavior therapy skills. Behaviour Research and Therapy, 47(11), 921–930. https://doi.org/10.1016/j.brat.2009.07.011CrossRefPubMed

Dorsey, S., Lyon, A. R., Pullmann, M. D., Jungbluth, N., Berliner, L., & Beidas, R. (2017). Behavioral rehearsal for analogue fidelity: feasibility in a state-funded children’s mental health initiative. Administration and Policy in Mental Health and Mental Health Services Research, 44(3), 395–404. https://doi.org/10.1007/s10488-016-0727-4CrossRefPubMed

Epstein, R. M. (2007). Assessment in medical education. New England Journal of Medicine, 356(4), 387–396. https://doi.org/10.1056/NEJMra054784CrossRef

Garland, A. F., Bickman, L., & Chorpita, B. F. (2010a). Change what? Identifying quality improvement targets by investigating usual mental health care. Administration and Policy in Mental Health and Mental Health Services Research, 37(1–2), 15–26. https://doi.org/10.1007/s10488-010-0279-yCrossRefPubMed

Garland, A. F., Hurlburt, M. S., Brookman-Frazee, L., Taylor, R. M., & Accurso, E. C. (2010b). Methodological challenges of characterizing usual care psychotherapeutic practice. Administration and Policy in Mental Health and Mental Health Services Research, 37(3), 208–220. https://doi.org/10.1007/s10488-009-0237-8CrossRefPubMed

Goulet, F., Jacques, A., Gagnon, R., Racette, P., & Sieber, W. (2007). Assessment of family physicians’ performance using patient charts. Evaluation & the Health Professions, 30(4), 376–392. https://doi.org/10.1177/0163278707307924CrossRef

Guerra, C. E., Jacobs, S. E., Holmes, J. H., & Shea, J. A. (2007). Are physicians discussing prostate cancer screening with their patients and why or why not? A pilot study. Journal of General Internal Medicine, 22(7), 901–907. https://doi.org/10.1007/s11606-007-0142-3CrossRefPubMedPubMedCentral

Heck, N. C., Saunders, B. E., & Smith, D. W. (2015). Web-based training for an evidence-supported treatment: Training completion and knowledge acquisition in a global sample of learners. Child Maltreatment, 20(3), 183–192. https://doi.org/10.1177/1077559515586569CrossRefPubMed

Hogue, A., Dauber, S., Henderson, C. E., & Liddle, H. A. (2014). Reliability of therapist self-report on treatment targets and focus in family-based intervention. Administration and Policy in Mental Health and Mental Health Services Research, 41(5), 697–705. https://doi.org/10.1007/s10488-013-0520-6CrossRefPubMed

Hogue, A., Henderson, C. E., Dauber, S., Barajas, P. C., Fried, A., & Liddle, H. A. (2008). Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems. Journal of Consulting and Clinical Psychology, 76(4), 544–555. https://doi.org/10.1037/0022-006X.76.4.544CrossRefPubMedPubMedCentral

Hukkelberg, S. S., & Ogden, T. (2013). Working alliance and treatment fidelity as predictors of externalizing problem behaviors in parent management training. Journal of Consulting and Clinical Psychology, 81(6), 1010–1020. https://doi.org/10.1037/a0033825CrossRefPubMed

Hurlburt, M. S., Garland, A. F., Nguyen, K., & Brookman-Frazee, L. (2010). Child and family therapy process: Concordance of therapist and observational perspectives. Administration and Policy in Mental Health and Mental Health Services Research, 37(3), 230–244. https://doi.org/10.1007/s10488-009-0251-xCrossRefPubMed

Imel, Z. E., Baldwin, S. A., Baer, J. S., Hartzler, B., Dunn, C., Rosengren, D. B., & Atkins, D. C. (2014). Evaluating therapist adherence in motivational interviewing by comparing performance with standardized and real patients. Journal of Consulting and Clinical Psychology, 82(3), 472–481. https://doi.org/10.1037/a0036158CrossRefPubMedPubMedCentral

Jennett, P., & Affleck, L. (1998). Chart audit and chart stimulated recall as methods of needs assessment in continuing professional health education. Journal of Continuing Education in the Health Professions, 18(3), 163–171. https://doi.org/10.1002/chp.1340180306CrossRef

Kilbourne, A. M., Neumann, M. S., Pincus, H. A., Bauer, M. S., & Stall, R. (2007). Implementing evidence-based interventions in health care: Application of the replicating effective programs framework. Implementation Science, 2(1), 42. https://doi.org/10.1186/1748-5908-2-42CrossRefPubMedPubMedCentral

McGrew, J. H., Bond, G. R., Dietzen, L., & Salyers, M. (1994). Measuring the fidelity of implementation of a mental health program model. Journal of Consulting and Clinical Psychology, 62(4), 670–678. https://doi.org/10.1037/0022-006X.62.4.670CrossRefPubMed

McLeod, B. D., & Weisz, J. R. (2005). The therapy process observational coding system-alliance scale: Measure characteristics and prediction of outcome in usual clinical practice. Journal of Consulting and Clinical Psychology, 73(2), 323–333. https://doi.org/10.1037/0022-006X.73.2.323CrossRefPubMed

McLeod, B. D., & Weisz, J. R. (2010). The therapy process observational coding system for child psychotherapy strategies scale. Journal of Clinical Child & Adolescent Psychology, 39(3), 436–443. https://doi.org/10.1080/15374411003691750CrossRef

McMillen, J. C., Hawley, K. M., & Proctor, E. K. (2016). Mental health clinicians’ participation in web-based training for an evidence supported intervention: Signs of encouragement and trouble ahead. Administration and Policy in Mental Health and Mental Health Services Research, 43(4), 592–603. https://doi.org/10.1007/s10488-015-0645-xCrossRefPubMed

Miller, P. A., Nayer, M., & Eva, K. W. (2010). Psychometric properties of a peer-assessment program to assess continuing competence in physical therapy. Physical Therapy, 90(7), 1026–1038. https://doi.org/10.2522/ptj.20080137CrossRefPubMed

Nakamura, B. J., Selbo-Bruns, A., Okamura, K., Chang, J., Slavin, L., & Shimabukuro, S. (2014). Developing a systematic evaluation approach for training programs within a train-the-trainer model for youth cognitive behavior therapy. Behaviour Research and Therapy, 53, 10–19. https://doi.org/10.1016/j.brat.2013.12.001CrossRefPubMed

National Crime Victims Research & Treatment Center. (2007). TF-CBT web: First year report. Medical University of South Carolina.

Perepletchikova, F., Hilt, L. M., Chereji, E., & Kazdin, A. E. (2009). Barriers to implementing treatment integrity procedures: Survey of treatment outcome researchers. Journal of Consulting and Clinical Psychology, 77(2), 212–218. https://doi.org/10.1037/a0015232CrossRefPubMed

Perepletchikova, F., Treat, T. A., & Kazdin, A. E. (2007). Treatment integrity in psychotherapy research: Analysis of the studies and examination of the associated factors. Journal of Consulting and Clinical Psychology, 75(6), 829–841. https://doi.org/10.1037/0022-006X.75.6.829CrossRefPubMed

Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38(2), 65–76. https://doi.org/10.1007/s10488-010-0319-7CrossRefPubMed

Puspitasari, A., Kanter, J. W., Murphy, J., Crowe, A., & Koerner, K. (2013). Developing an online, modular, active learning training program for behavioral activation. Psychotherapy, 50(2), 256–265. https://doi.org/10.1037/a0030058CrossRefPubMed

Rodriguez-Quintana, N., & Lewis, C. C. (2018). Observational coding training methods for CBT treatment fidelity: A systematic review. Cognitive Therapy and Research, 42(4), 358–368. https://doi.org/10.1007/s10608-018-9898-5CrossRef

Rozek, D. C., Serrano, J. L., Marriott, B. R., Scott, K. S., Hickman, L. B., Brothers, B. M., Lewis, C. C., & Simons, A. D. (2018). Cognitive behavioural therapy competency: Pilot data from a comparison of multiple perspectives. Behavioural and Cognitive Psychotherapy, 46(2), 89. https://doi.org/10.1017/S1352465817000662CrossRef

Schoenwald, S. K., Carter, R. E., Chapman, J. E., & Sheidow, A. J. (2008). Therapist adherence and organizational effects on change in youth behavior problems one year after multisystemic therapy. Administration and Policy in Mental Health and Mental Health Services Research, 35(5), 379–394. https://doi.org/10.1007/s10488-008-0181-zCrossRefPubMed

Schoenwald, S. K., & Garland, A. F. (2013). A review of treatment adherence measurement methods. Psychological Assessment, 25(1), 146–156. https://doi.org/10.1037/a0029715CrossRefPubMed

Sholomskas, D. E., Syracuse-Siewert, G., Rounsaville, B. J., Ball, S. A., Nuro, K. F., & Carroll, K. M. (2005). We don’t train in vain: A dissemination trial of three strategies of training clinicians in cognitive-behavioral therapy. Journal of Consulting and Clinical Psychology, 73(1), 106–115. https://doi.org/10.1037/0022-006X.73.1.106CrossRefPubMedPubMedCentral

Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428. https://doi.org/10.1037/0033-2909.86.2.420CrossRefPubMed

Southam-Gerow, M. A., Bonifay, W., McLeod, B. D., Cox, J. R., Violante, S., Kendall, P. C., & Weisz, J. R. (2020). Generalizability and decision studies of a treatment adherence instrument. Assessment, 27(2), 321–333. https://doi.org/10.1177/1073191118765365CrossRefPubMed

Southam-Gerow, M. A., McLeod, B. D., Arnold, C. C., Rodríguez, A., Cox, J. R., Reise, S. P., Bonifay, W. E., Weisz, J. R., & Kendall, P. C. (2016). Initial development of a treatment adherence measure for cognitive-behavioral therapy for child anxiety. Psychological Assessment, 28(1), 70–80. https://doi.org/10.1037/pas0000141CrossRefPubMed

Waltz, J., Addis, M. E., Koerner, K., & Jacobson, N. S. (1993). Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence. Journal of Consulting and Clinical Psychology, 61(4), 620–630. https://doi.org/10.1037/0022-006X.61.4.620CrossRefPubMed

Webb, C. A., DeRubeis, R. J., & Barber, J. P. (2010). Therapist adherence/competence and treatment outcome: A meta-analytic review. Journal of Consulting and Clinical Psychology, 78(2), 200. https://doi.org/10.1037/A0018912CrossRefPubMedPubMedCentral

Titel: Role-Play Assessment of Therapist Adherence and Skill in Implementation of Trauma-Focused Cognitive-Behavioral Therapy
verfasst von: Brigid R. Marriott
Evelyn Cho
Siena K. Tugendrajch
Matthew D. Kliethermes
J. Curtis McMillen
Enola K. Proctor
Kristin M. Hawley
Publikationsdatum: 21.09.2021
Verlag: Springer US
Erschienen in: Administration and Policy in Mental Health and Mental Health Services Research / Ausgabe 3/2022
Print ISSN: 0894-587X
Elektronische ISSN: 1573-3289
DOI: https://doi.org/10.1007/s10488-021-01169-9

Update Psychiatrie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.

Newsletter bestellen

Live-Webinar "Urologie und Sexualmedizin in der Praxis"

Springer Medizin

Role-Play Assessment of Therapist Adherence and Skill in Implementation of Trauma-Focused Cognitive-Behavioral Therapy

Abstract

Neu im Fachgebiet Psychiatrie

Hörschwäche erhöht Demenzrisiko unabhängig von Beta-Amyloid

So wirken verschiedene Alkoholika auf den Blutdruck

„Übersichtlicher Wegweiser“: Lauterbachs umstrittener Klinik-Atlas ist online

ADHS-Medikation erhöht das kardiovaskuläre Risiko

Update Psychiatrie

Live-Webinar "Urologie und Sexualmedizin in der Praxis"

Springer Medizin

Abstract

Bitte loggen Sie sich ein, um Zugang zu diesem Inhalt zu erhalten

Weitere Artikel der Ausgabe 3/2022

Socioeconomic Deprivation and Dropout from Contemporary Psychological Intervention for Common Mental Disorders: A Systematic Review

Trial-Based Economic Evaluations of Supported Employment for Adults with Severe Mental Illness: A Systematic Review

A Review of Self-Compassion as an Active Ingredient in the Prevention and Treatment of Anxiety and Depression in Young People

Knowledge and Attitudes Toward an Artificial Intelligence-Based Fidelity Measurement in Community Cognitive Behavioral Therapy Supervision

A Randomized Trial of Specialty Mental Health Probation: Measuring Implementation and Effectiveness Outcomes

Mental Health Information Seeking Online: A Google Trends Analysis of ADHD

Neu im Fachgebiet Psychiatrie

Hörschwäche erhöht Demenzrisiko unabhängig von Beta-Amyloid

So wirken verschiedene Alkoholika auf den Blutdruck

„Übersichtlicher Wegweiser“: Lauterbachs umstrittener Klinik-Atlas ist online

ADHS-Medikation erhöht das kardiovaskuläre Risiko

Update Psychiatrie