Abstract
Background
Given dismal attendance rates in community-based care for children and families, it is critical that evidence-informed attendance engagement strategies be implemented within community service systems. There is growing research on effective methods for training in evidence-based practices (EBPs), and one method that shows promise is the learning collaborative modeled after the Institute for Healthcare Improvement’s Breakthrough Series Collaborative framework.
Objective
This study examines implementation outcomes of a learning collaborative based on the Breakthrough Series Collaborative that was conducted to improve attendance engagement in community-based early childhood intervention programs using evidence-informed strategies.
Methods
A total of 29 providers from four programs within a large regional hospital participated. Qualitative and quantitative data collected prior, during, and at the completion of the 9-month learning collaborative as part of a process evaluation. Data were analyzed to examine the feasibility, acceptability, adoption and fidelity, and planned sustainability of strategies to facilitate attendance engagement as a result of the learning collaborative.
Results
Results indicate that: (1) using a learning collaborative implementation method with early intervention providers was feasible; (2) the method was acceptable based on perceived improvements in attendance and a significant increase in attitudes towards EBPs; (3) the method supported successful self-reported adoption and fidelity of engagement strategies; and (4) the method facilitated planned sustainability of practice changes.
Conclusions
The learning collaborative can be a useful implementation strategy within early childhood intervention programs to promote the use of EBPs, including enhancing attendance engagement through evidence-informed strategies.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aarons, G. A. (2005). Measuring provider attitudes toward evidence-based practice: Consideration of organizational context and individual differences. Child and Adolescent Psychiatric Clinics of North America, 14, 255–271.
Aarons, G. A., Ehrhart, M. G., R., Fahranak L., & Hurlburt, M. (2013). Leadership and organizational change for implementation (LOCI): A mixed-method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Manuscript submitted for publication.
Aarons, G. A., Hurlburt, M., & Horwitz, S. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38, 4–23.
Bannon, W. M., & McKay, M. (2005). Are barriers to service and parental preference match for service related to urban child mental health service use? Families in Society, 86, 30–34.
Borntrager, C. F., Chorpita, B. F., Higa-McMillan, C., & Weisz, J. R. (2009). Provider attitudes toward evidence-based practices: Are the concerns with the evidence or with the manuals? Psychiatric Services, 60, 677–681.
Brookman-Frazee, L., Stahmer, A. C., Lewis, K., Feder, J. D., & Reed, S. (2012). Building a research-community collaborative to improve community care for infants and toddlers at-risk for autism spectrum disorders. Journal of Community Psychology, 40, 715–734.
Cavaleri, M. A., Franco, L. M., McKay, M. M., Appel, A., Bannon, W. M., Jr, Bigley, M. F., et al. (2007). The sustainability of a learning collaborative to improve mental health service use among low-income urban youth and families. Best Practices in Mental Health, 3, 52–61.
Cavaleri, M. A., Gopalan, G., McKay, M. M., Appel, A., Bannon, W. M., Jr, Bigley, M. F., et al. (2006). Impact of a learning collaborative to improve child mental health service use among low-income urban youth and families. Best Practices in Mental Health, 2, 67–79.
Cavaleri, M. A., Gopalan, G., McKay, M. M., Messam, T., Velez, E., & Elwyn, L. (2010). The effect of a learning collaborative to improve engagement in child mental health services. Children and Youth Services Review, 32, 281–285.
Deo, S., McInnes, K., Corbett, C. J., Landon, B. E., Shapiro, M. F., Wilson, I. B., et al. (2009). Associations between organizational characteristics and quality improvement activities of clinics participating in a quality improvement collaborative. Medical Care, 47, 1026–1030.
Duffy, F., Chung, H., Trivedi, M., Rae, D. S., Reiger, D. A., & Katzelnick, D. J. (2008). Systematic use of patient-rated depression severity monitoring: Is it helpful and feasible in clinical psychiatry? Psychiatric Services, 59, 1148–1154.
Ebert, L., Amaya-Jackson, L., Markiewicz, J. M., Kisiel, C., & Fairbank, J. A. (2012). Use of the Breakthrough Series Collaborative to support broad and sustained use of evidence-based trauma treatment for children in community practice settings. Administration and Policy in Mental Health and Mental Health Services, 39, 187–199.
Epstein, J. N., Langberg, J. M., Lichtenstein, P. K., Kolb, R. C., & Stark, L. J. (2010). Sustained improvement in pediatricians’ ADHD practice behaviors in the context of a community-based quality improvement initiative. Children’s Health Care, 39, 296–311.
Epstein, J. N., Langberg, J. M., Lichtenstein, P. K., Mainwaring, B. A., Luzader, C. P., & Stark, L. J. (2008). Community-wide intervention to improve the attention-deficit/hyperactivity disorder assessment and treatment practices of community physicians. Pediatrics, 122, 19–27.
Glisson, C., Hemmelgarn, A., Green, P., & Williams, N. J. (2013). Randomized trial of the availability, responsiveness and continuity (ARC) organizational intervention for improving youth outcomes in community mental health programs and clinicians serving youth. Journal of the American Academy of Child and Adolescent Psychiatry, 51, 780–787.
Glisson, C., Schoenwald, S. K., Hemmelgarn, A., Green, P., Dukes, D., Armstrong, K. S., et al. (2010). Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. Journal of Consulting and Clinical Psychology, 78, 537–550.
Gopalan, G., Goldstein, L., Klingenstein, K., Sicher, C., Blake, C., & McKay, M. M. (2010). Engaging families into child mental health treatment: Updates and special considerations. Journal of the Canadian Academy of Child And Adolescent Psychiatry, 19, 182–196.
Herschell, A., Kolko, D., Baumann, B. L., & Davis, A. C. (2010). The role of therapist training in the implementation of psychosocial treatments: A review and critique with recommendations. Clinical Psychology Review, 30, 448–466.
Hulscher, M. E., Schouten, L. M., Grol, R. P., & Buchan, H. (2013). Determinants of success of quality improvement collaboratives: What does the literature show? BMJ Quality & Safety, 22, 19–31.
Individuals with Disabilities Education Act. (2004). 20 U.S.C. §1400–1482.
Ingoldsby, E. M. (2010). Review of interventions to improve family engagement and retention in parent and child mental health programs. Journal of Child and Family Studies, 19, 629–645.
Institute for Healthcare Improvement. (2003). The breakthrough series: IHI’s collaborative model for achieving breakthrough improvement. Boston: Institute for Healthcare Improvement.
Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press.
Kilo, C. M. (1998). A framework for collaborative improvement: Lessons from the Institute for Healthcare Improvement’s breakthrough series. Quality Management in Health care, 6, 1–14.
Lopez, M. A., Osterberg, L. D., Jensen-Doss, A., & Rae, W. A. (2011). Effects of workshop training for providers under mandated use of an evidence-based practice. Administration and Policy in Mental Health and Mental Health Services Research, 38, 301–312.
Lyon, A. R., Stirman, S. W., Kerns, S. E. U., & Bruns, E. J. (2011). Developing the mental health workforce: Review and application of training approaches from multiple disciplines. Administration and Policy in Mental Health and Mental Health Services Research, 38, 238–253.
McKay, M. M., McCadam, K., & Gonzales, J. J. (1996a). Addressing the barriers to mental health services for inner city children and their caretakers. Community Mental Health Journal, 33, 353–361.
McKay, M. M., Nudelman, R., McCadam, K., & Gonzales, J. (1996b). Involving inner-city families in mental health services: First interview engagement skills. Research on Social Work Practice, 6, 462–472.
McKay, M. M., Pennington, J., Lynn, C. J., & McAdams, K. (2001). Understanding urban child mental health service use: Two studies of child, family, and environmental correlates. Journal of Behavioral Health Services & Research, 28, 475–483.
McKay, M. M., Stoewe, J., McCadam, K., & Gonzales, J. (1998). Increasing access to child mental health services for urban children and their caretakers. Health and Social Work, 23, 9–15.
Mittman, B. S. (2004). Creating the evidence base for quality improvement collaboratives. Annals of Internal Medicine, 140, 897–901.
Nelson, T. D., & Steele, R. G. (2007). Predictors of practitioner self-reported use of evidence-based practices: Practitioner training, clinical setting, and attitudes toward research. Administration and Policy in Mental Health and Mental Health Services Research, 34, 319–330.
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G. A., Bunger, A., et al. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38, 65–76.
Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: Free Press.
Roosa, M., Scripa, J. S., Zastowny, T. R., & Ford, J. H., II. (2011). Using a NIATx based local learning collaborative for performance improvement. Evaluation and Program Planning, 34, 390–398.
Rutkowski, B. A., Gallon, S., Rawson, R. A., Freese, T. E., Bruehl, A., Crevecoeur-MacPhail, D., et al. (2010). Improving client engagement and retention in treatment: The Los Angeles County experience. Journal of Substance Abuse Treatment, 39, 78–86.
Schouten, L. M., Hulscher, M. E., van Everdingen, J. J., Huijsman, R., & Grol, R. P. (2008). Evidence for the impact of quality improvement collaboratives: Systematic review. BMJ. British Medical Journal, 336, 1491–1494.
Solberg, L. I., Fischer, L. R., Wei, F., Rush, W. A., Conboy, K. S., Davis, T. F., et al. (2001). A CQI intervention to change the care of depression: A controlled study. Effective Clinical Practice, 4, 239–249.
Stahmer, A. C., & Aarons, G. A. (2009). Attitudes toward adoption of evidence-based practices: A comparison of autism early intervention providers and children’s mental health providers. Psychological Services, 6, 223–234.
Stahmer, A. C., Collings, N. M., & Palinkas, L. A. (2005). Early intervention practices for children with autism: Descriptions from community providers. Focus on Autism & Other Developmental Disabilities, 20, 66–79.
Strating, M. M. H., Broer, T., van Rooijen, S., Bal, R. A., & Nieboer, A. P. (2012). Quality improvement in long-term mental health: Results from four collaboratives. Journal of Psychiatric and Mental Health Nursing, 19, 379–388.
Strating, M. M. H., Nieboer, A. P., Zuiderent-Jerak, T., & Bal, R. A. (2011). Creating effective quality-improvement collaboratives: A multiple case study. BMJ Quality and Safety in Health Care, 20, 344–350.
Vannoy, S. D., Mauer, B., Kern, J., Girn, K., Ingoglia, C., Campbell, J., et al. (2011). A learning collaborative of CMHCs and CHCs to support integration of behavioral health and general medical care. Psychiatric Services, 62, 753–758.
Versteeg, M. H., Laurant, M. G. H., Franx, G. E., Jacobs, A. J., & Wensing, M. J. P. (2012). Factors associated with the impact of quality improvement collaboratives in mental helathcare: An exploratory study. Implementation Science, 7, 1–11.
Acknowledgments
Research reported in this publication was supported by NIMH of the National Institutes of Health under award number K23MH080149 (RHS) and K23MH077584 (LBF). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The intervention and learning collaborative curricula discussed herein (Training for the Engagement of Families or TIES; www.tiesengagement.com) were developed under a Small Business Innovation Research Contract (SBIR) funded by the National Institute of Mental Health, National Institute of Health, Department of Health and Human Services (HHSN271200800044C). This effort was based, to a large degree, on the work of Dr. Mary McKay of Mt. Sinai School of Medicine and the New York University Silver School of Social Work in collaboration with Danya International. The authors would like to acknowledge the First 5 Commission of San Diego for funding the TIES intervention training and learning collaborative. We would like to thank the four programs within Rady Children’s Hospital-San Diego’s Division of Developmental Services that participated in the learning collaborative: Children’s Care Connection, the Developmental Screening and Enhancement Program, Healthy Development Services North Central and North Coastal Care Coordination, and KidSTART. Thank you also to Lynn Eldred, Ed.D. for her thoughtful comments on an earlier version of the manuscript and to Dr. McKay for her consultation during the learning collaborative and assistance with data analysis.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Haine-Schlagel, R., Brookman-Frazee, L., Janis, B. et al. Evaluating a Learning Collaborative to Implement Evidence-Informed Engagement Strategies in Community-Based Services for Young Children. Child Youth Care Forum 42, 457–473 (2013). https://doi.org/10.1007/s10566-013-9210-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10566-013-9210-5