Structure and function of influence networks
Analysis of interview transcripts revealed that systems leaders develop and maintain networks of information and advice according to position in agency (e.g., directors, program managers), responsibility (probation, mental health, child welfare), geography (within a county, neighboring counties), and friendship ties (co-workers, classmates). These networks expose leaders to information about EBPs and opportunities to adopt EBPs and influence decisions to adopt EBPs. This information comes from others within the same county, including supervisors or employees within the same agency, counterparts in other agencies, community-based providers, and community advocates.
Noting both in-county and out-of-county resources for discussing EBPs was common across interviews. Within counties, participants said they drew on advice from individuals in their own agency (although this was not supported to a high degree by network analyses), outside agencies, community-based organizations, and community advocacy organizations. Network members located outside the county included professional organizations like the California Parole Officers Association, the Child Welfare Directors Association, and the California Mental Health Directors Association; intermediaries like the California Institute of Mental Health (CIMH); nonprofit foundations like the Annie E. Casey Foundation and Casey Family Programs; universities; and consultants. Peers from other counties were also an important source of information and advice; however, this occurred more in small rural counties than in large urban counties.
Among the forums for the exchange of information and advice about EBPs are regularly scheduled meetings within the county, region, and state; initiatives that involve contact of systems leaders by CIMH; agency staff; and other county agencies and community-based organizations. One director specifically cited a monthly statewide gathering as a particularly useful venue for gathering information on EBPs:
"I go monthly to the Children's System of Care meeting in Sacramento. And that's where other people in similar administrative positions to myself who are responsible for children's mental health services, we chew on these kinds of things. We discuss these kinds of things. And, you know, we have presentations, and so forth. So that is my peer group. And that, um, certainly provides a lot of information to me in making decisions."--Mental Health Department Director
Systems leaders also obtain information and advice on EBPs from counterparts in counties widely regarded for serving as "models" for innovation and EBP implementation, as one agency director noted when asked who she looks to outside her own county:
"There's a...always [our practice of] checking with Orange County, LA, [when considering adopting a new program]. Although quite big, they do some very progressive things as well. Um, and so you know which counties are kind of doing some leading edge, and, not just leading edge, but that also have uh, the evaluation component of it."--Chief Probation Officer
Participants described a wide range of advice seeking in qualitative interviews, which included both whether to implement an EBP (MTFC in particular) or a new, innovative program in their county and how to best implement such a program. Social-network survey-based ties between respondents included both types of advice seeking. While some participants in the qualitative interviews simply provided a name of someone who they had contacted about an EBP (or other program), others provided a more elaborate description of the advice-seeking interaction. For example, several participants discussed advice seeking in relation to the cost and feasibility of implementing a particular program; this included instances of where they had decided not to implement a specific program because they had been informed by their counterparts in other counties or directors of community-based organizations within their own county that the cost of implementation would be prohibitive. Others discussed advice seeking related to approaching appropriate community partners for collaboration.
Representations of the influence networks for exchanging information related to EBPs in general are found in Figure
1. Grey nodes represent individuals who reported being in implementation stages 0-1, blue-green nodes represent stages 2-6, and bright green nodes represent stages 7-8. White nodes depict individuals about whom insufficient information was obtained to ascertain implementation stage or about whom implementation stage is not relevant, such as individuals who work for CIMH or other non-county-affiliated organizations. A simple visual inspection of the network diagram reveals that many of the nodes in this network are connected to others in similar implementation stages.
Table
2 provides metrics that help to describe this network. A total of 176 individuals with 233 ties comprised this network. Network density was relatively low; less than one percent of all possible ties among nodes were present. We caution against over-interpreting this metric because mathematically, as network size increases, density decreases [
29]. Several other metrics suggested evidence of connectivity. There were eight unique components, that is, "disconnected" sub-networks. One of these components contained 81% of the overall network, while the remaining seven components ranged in size from one to nine individuals. Individuals from 10 of the 12 counties were represented in the largest component, and three counties were each represented in two or more components. Moreover, the average number of ties separating any two individuals in the network was 1.9.
Table 2
Network metrics for combined interview and survey network (n = 176)
Size | 176 |
Number of ties | 223 |
Density | 0.0072 |
Average distance | 1.884 |
Number of components | 8 |
In-degree centrality | 1.27 (0.91) |
Out-degree centrality | 1.27 (3.05) |
The principle of homophily was well supported for both county and implementation stage among members of the original sample. On average, 81% of network ties were among individuals who came from the same county, and 83% of network ties were among individuals who were classified in the same implementation stage as the respondent. Interestingly, only 38% of network ties were among individuals who came from the same county agency as the respondent. Taken together, these results indicate that individuals often rely on others from within their own county for advice on EBPs, although not necessarily individuals from within their agency, and from individuals outside their county. This latter observation was supported by the fact that seven counties had links to one individual who works for the CIMH and is known throughout the state as someone on whom agency directors can rely for information about EBPs.
Implementation stage was also associated with position in the overall advice network. The multivariate regression model presented in Table
3 reveals that county-level and network-position specific variables were important independent correlates of implementation stage. Individuals in large counties, relative to small, reported higher implementation stage, and urban counties, relative to rural, reported higher implementation stage. Increasing in-degree centrality was positively associated with implementation stage at the two-year follow-up, while out-degree centrality was not. These latter results indicate that, adjusting for county-level attributes, being nominated more frequently by others in the network was positively associated with implementation stage two years later, while the number of nominations an individual provided were not associated with implementation stage.
Table 3
Regression of implementation stage on centrality, adjusting for county size and urban/rural classification (n = 137)
In-degree centrality | 0.16 | 0.07 | 2.26 | .03 |
Out-degree centrality | 0.01 | 0.02 | 0.61 | .54 |
Large county | 0.43 | 0.14 | 3.14 | .00 |
Urban county | 0.47 | 0.15 | 3.24 | .00 |
Collaboration as critical to evidence-based practice implementation
In addition to identifying the potential predictors of implementation stage and supplementing the construction of the social networks, the qualitative analysis of the semistructured interviews identified features of these networks that were critical to the process of EBP implementation. Perhaps the most salient of these features was the role of collaboration within and between counties. Within counties, single agencies often lacked resources to implement EBPs independently and noted that implementation requires good systems partners. In small, rural counties where agencies have limited resources to implement EBPs on their own, agency directors cited a desire to participate in the Cal-40 Study in clusters with neighboring counties.
Poor history of collaboration was often cited as a reason for failure to implement EBPs. The reasons for the lack of collaboration identified by study participants included the following: lack of funding to support a collaboration, different priorities and mandates of the collaborating agencies, different organizational cultures of the collaborating agencies and the lack of understanding of these cultures, and differences in personality and the strained relationships caused by these differences.
Finally, criteria for effective collaborations among agencies in public youth-serving agencies included individuals who can play key roles in the collaborative process, especially agency directors and administrators with knowledge or experience working for another agency who can serve as a collaboration broker or facilitator. For example, one participant cited her varied experience working for multiple agencies as beneficial to understanding complex system interactions, stating, "I fortunately have had the experience of being a probation officer, a social service worker, and a mental health clinician" (Mental Health Department Child/Adolescent Program Chief).
Role of influence networks in MTFC implementation
These information and advice networks appear to have played an important role in the implementation of MTFC among the first cohort of counties participating in the Cal-40 Study. For those who had agreed to participate or were considering participation at the time they were interviewed, information about MTFC and the Cal-40 Study was obtained from presentations given by CIMH representatives at state or regional meetings, direct contact by CIMH with county agency directors, direct contact by other agency directors within the county, or staff within the agency:
"It came to my attention two different ways. I started hearing some discussion about it at the small county association meetings, which is a break off of the full body county Mental Health Directors Association. And I heard it from one or two of my peers... Then, the newest program manager brought it to my attention. And I think she found it on the CIMH website..."--Mental Health Program Director
Only one of the seven systems leaders interviewed from the three counties that had either decided not to participate in the Cal-40 Study or had not advanced beyond stage 1 had received any information about MTFC or the Cal-40 Study.