Exploring multi-level system factors facilitating educator training and implementation of evidence-based practices (EBP): a study protocol

The Centers for Disease Control (CDC) estimates that 1 in 68 children have an autism spectrum disorder (ASD; [1]). Long-term outcomes for this population are poor [2‐5], and the annual cost in the USA is estimated to be $268 billion [6]. The service system accessed most often for school-age children with ASD is education (Brookman-Frazee, Baker-Ericzen, Stahmer, Mandell, Haine, and Hough [7]). The number of children with ASD served by schools has grown fivefold from 93,000 in 2000 to 455,000 in 2011. The education system is responsible for targeting a wide range of needs that interfere with a child’s ability to benefit from general education including improving learning skills such as attention and engagement and core symptoms of ASD.

The federal legislation known as “Every Child Succeeds,” passed in 2015 [8], and the Individuals with Disabilities Education Act [9], both specify that the practices used in schools must be those supported by scientifically based evidence. Additionally, specific evidence-based practices (EBPs) for ASD have been identified for school personnel [10‐13]. However, research indicates a large gap between research and school practices [14] with EBP for ASD not being routinely used in schools [15, 16]. Similar to gaps in education, generally in ASD, teachers and other school-based providers can learn to use EBPs when trained by experts [17] but do not typically incorporate EBPs into school-based programs over the long term [18‐20]. This is problematic because children with ASD show significant gains when they receive services implemented with high fidelity and when schools do not use interventions with proven efficacy; the courts can require them to provide costly alternatives, such as increased one-to-one intervention or placement in a private school [21, 22].

Scaling up interventions across multiple schools, districts and statewide present an additional challenge. Most state systems have a very limited capacity for scaling up interventions in ways that lead to meaningful improvements in outcomes for students [23]. Based on the research in mental health systems, it is clear that facilitation of effective EBP use requires leadership coordination and support at the school, district, and state levels; however, leaders at each of these levels have a very limited understanding of the factors beyond their immediate locale, such as system-wide policies and support, that may lead to gaps in the use of EBP in special education. While recent systematic reviews and meta-analyses of care coordination consistently report differences (of a moderate to medium effect size) in outcomes of mental health intervention and physical conditions management programs when collaborative practices within systems are compared to usual care [24‐26], the “active ingredients” that allow for systematic implementation of coordinated care across systems remain obscure. There are critical gaps in evidence about the structure, function, and benefits of team coordination across multiple systems.

School-based service for special education involves a team of providers and administrators to account for the complexity of care needed for this population of students. For example, California has 1028 school districts divided into 122 regional consortiums for the purpose of providing for all special education service needs of children in the regions. These Special Education Local Plan Areas (SELPAs) work alongside school district staff to meet the children’s educational and mental health needs. They offer varying levels of support and structure from taking an active role in effective implementation of EBPs by providing autism specialists to support training and implementation to simply providing oversight for compliance. High-level administrators at the SELPA and school district levels are responsible for the provision of educational and associated programming, including staffing allocation, curricula, and use of resources. Administrators report that scientific evidence does not readily affect their programming decisions [27]. Administrators at both the district and SELPA level likely have a strong influence on the culture and climate related to the use of EBP as well as resource allocation for training, materials, and supervision in the use of new practices. Of course, principals play an important role in the leadership of an individual school site. However, only a small percentage of principals was special educators prior to their administrative roles and have limited professional training in special education [28]. Additionally, preliminary data indicate teachers and other high-level school district staff perceive special education directors, mid-level specialists, and teachers as key decision-makers across phases of implementation for ASD programs as compared to school site principals [29]. Teachers and paraprofessionals are front-line EBP users. They are the ones to adapt and modify practices and to determine the use. At each level of administration, resource allocation, implementation climate, education, experience with ASD, attitudes toward EBP, and collaboration have the potential to affect EBP use. Therefore, systems and processes at multiple levels must be considered for implementation in special education.

Identifying malleable factors that influence the implementation of EBP across system levels for students with ASD will help policymakers and administrators make system-wide changes to positively impact teacher training and effective use of EBP both for ASD and more generally. Characterization of malleable organizational factors with accompanying linkage to implementation and clinical outcomes is an innovative process with the potential to increase understanding of the mechanisms of action of system level implementation factors. The CAPTAIN statewide implementation effort provides a natural vehicle for examining facilitators and barriers to training and support for community providers. Outcomes will inform scale-up efforts for EBP implementation broadly and implementation science methods.

In addition, to our knowledge, no one has used formal social network analysis to capture multi-level collaboration among state-, district-, school-, and classroom-level stakeholders in the field of ASD. The proposed research will advance our scientific understanding of social determinants of treatment and outcomes for children with ASD and provide new conceptual tools for understanding disparities in treatment and outcomes. The proposed study both compliments and redirects existing research by considering the configuration of the social networks of EBP supporters and providers across system levels, overcoming previous methodological barriers that constrain service research to only consider one level per study. In addition to new approaches in research, the proposed project provides the basis for new social network interventions in autism that strengthen the capacity of providers to work together to provide higher quality EBP for children with ASD.

Findings will add to the very limited research examining cross-level mechanisms linking both inner and outer context factors to changes in provider behaviors [31] and to consumer outcomes. Data from this exploratory project will be used to develop an implementation assessment intervention package that will address malleable factors identified as important to the teacher and student outcomes. This project may provide a generalizable model for the development of multi-level implementation interventions across complex service settings.