We interviewed 14 individuals (five at unit ‘A’, five at unit ‘B’, and four at unit ‘C’). They were managers or unit leaders (n = 5), EIDM experts such as project specialists or health promotion consultants who helped staff with the EIDM process (n = 5), and four others (2 nurses, 1 epidemiologist, and 1 librarian). Half of the sample was central actors (fourth quartile of indegree centrality) in information-seeking and expertise networks. Six worked in a supervisory/administrative division, and the others in practice-based divisions such as family health, chronic diseases, and environmental health.
Staff engagement
The senior managers of each health unit invited a group of staff to participate in trainings and subsequent workgroups. At unit A, 51 staff members (8% of 638 total workforce), at unit B, thirteen staff (1% of 1068 total workforce), and at unit C, 18 (9% of 201 total workforce) participated in the intervention. Among the most central actors in information seeking and expertise networks 61% at unit A (mostly managers and project specialists), 10% at unit B (mostly health promotion consultants), and 56% at unit C (mostly program managers) participated in the intervention. Four of 5 epidemiologists at unit C who were central in information sharing networks did not engage in the intervention.
Interviewees in three units identified several factors affecting the process of engagement, which were classified into the role of
leadership support,
relevance of staff’s roles to EIDM, and
non-participatory engagement:
In the three health units the decision to participate in the study, the level and breadth of engagement, and the mechanism of staff recruitment in the intervention, was mainly a top-down mechanism initiated and supported by each unit’s organizational leaders and/or divisional managers.
The leaders of public health units (such as the medical officers of health-MOH) were potential initiators and champions of this process. This role was much more prominent at unit A, probably due to the charismatic character of its leaders, as explained by a project specialist: “I think [the leader’s] style is to commandeer the resources that she needs. I sometimes get the impression that what [the organization head] wants she gets, in terms of staff time or resources or whatever.”(2-A)
The strong message given by the units’ leaders was very effective in motivating the staff to participate, as explained by a manager at unit C: “a message from MOH; knowing that EIDM was a priority, and I think he had sent those messages to the staff a number of times. And so all staff in the department knew, and he often would bring it up whenever he could.”(1-C)
The role of the leader in the process of implementation was not mentioned in the interviews at unit B where decisions about study recruitment and the level of involvement was more localized at the organizational division level. Some divisions had a high participation rate and others refused to participate. As explained by a manager at unit B,
“What we had is it was really left up to different [divisions]
to set their own level of involvement, and many of them sadly did not pick up the opportunity”(1-B).
Program managers of health units selected the staff whose professional roles they considered relevant to EIDM. The composition of the selected group differed across health units depending on each unit’s organizational structure and how the leaders viewed EIDM in relation to staff roles. A leader at unit ‘A’ explained her selection process this way: “We chose participants by the roles in the organization. Every specialist, every supervisor, every manager is eligible to participate. And we have systematically tried to enlist every single one.”(4-A).
At unit ‘B’, health promotion consultants were highly represented in the selected group. However, due to their diverse backgrounds and broad definition of their roles, these consultants differed considerably in terms of their expertise in EIDM and its perceived relevance to their jobs, as explained by one consultant:“Staff … kept saying, ’This is irrelevant to us, I have done this in my Masters’. We had such a hard time finding staff who do this kind of work; they were very resistant”. (4-B)
The role of consultant was defined broadly at unit B, and could include roles that were not related to EIDM. However, all consultants were invited to participate in the intervention.
At unit ‘C’ nurses were the main group recruited by program managers based on their prior experience with the health problems to be addressed in evidence-based reports; as explained by a manager:
It would be very hard to [select trainees] otherwise because they are assigned to specific work. So if I had a staff who was not assigned to that work, that means she has less time to do her other work… because we just don't have enough resources. (1-C).
Staff showed resistance when they found the intervention, and EIDM in general, irrelevant to their roles and job definition, as explained by a manager at unit C:
I found the process somewhat complicated even though I know it doesn’t necessarily meant to be, but I think the way we were seeing is that: here is the main work that we have to do and here is the process, separate, while really they should be integrated. (4-C).
Compared to unit ‘A’, it seems that expectations from managers at units ‘B’ and ‘C’ were not as clear, in part because managers were seen as overseeing the production of evidence-based reports by nurses, rather than getting involved in their development, as explained by a manager in unit ‘C’: “[the recruitment process] wasn't really saying all managers have to participate in the intervention.”(1-C)
Interestingly, epidemiologists (who hold a central position in information-seeking networks) were not invited to join the intervention at unit ‘C’. One manager explained this as follows:
“ To be honest, I never thought of involving them. I thought we were supposed to keep it within our divisions.”(4-C). This disconnect was further reinforced by the epidemiologists’ belief that their job did not entail working with research evidence because their roles were more evaluation-oriented and
“tied to the processes” (2-C) [indirect experience of the interviewee].
At unit A, where the leaders’ involvement and interest was most prominent among the three units, the strong message by leaders asking for participation resulted in perception by some that “staff were kind of forced to go to the workshops” (3-A). This, in turn, negatively impacted staff’s motivation, as explained by a librarian at unit A: “You didn’t have a choice. [The] organization expected you to do so. It was like force feeding the staff.”(3-A).
A recurring theme in the interviews that appeared most frequently with participants from unit B was that at the beginning of the study the staff were not fully aware of what the intervention was about, how evidence would be useful in their practice, why they had been chosen, and what were they supposed to do with whatever they would learn in the training. As explained by a consultant: “I kind of found we were really in the dark, I just got an email [that said], ‘You are coming to this meeting so you are gonna look at this topic area’…and personally when this is over I don’t even know what is gonna happen.”(5-B) Another consultant at unit B explained about the miscommunication between the managers and staff regarding the aims of the study and the reasons for choosing certain staff to participate: “So they picked a whole bunch of health promotion consultants. So I remember a consultant … who wasn’t even told why she was there… It was not marketed like: ‘here is this initiative; who is interested?’ It was like: ‘you have been selected’.”(4-B)
Communication during trainings
After the workshop, staff were assigned to evidence-based report development teams. At unit ‘A’ the evidence review teams regularly held progress meetings moderated by the KB and the organizational leader. At units B and C, the progress meetings were more localized and limited to each work group and KB. At unit A, KB served onsite with regular office hours, but in two other units her engagement was a combination of onsite and offsite consultation. Quantitative analysis showed an increasing tendency among engaged staff to form information sharing clusters [
20]. The interviewees explained about the dynamics of
communications among participants and
communications with EIDM experts:
One result of these frequent interactions among the engaged staff was the ability to observe each other’s progress and learn from their experience, as explained by a project specialist: “[For] six to eight months we were meeting every couple of weeks. So we were hearing what other people’s projects are, and watch them struggle, and think, ‘Oh! They find synthesis just as difficult [to do as I do]’”(2-A); and echoed by an organizational leader: “People often would say at the end of meeting: ’Oh! I found it so interesting that such and so were having this problem because that was my problem too.’ And there was a lot of identification with other people’s process and experience” (4-A).
Communication among the groups was more sporadic in units ‘B’ and ‘C’, mostly limited to the separate meetings of evidence-based report teams and KB, as described by a manager at unit ‘C’: “
We had regular meetings with [KB]
… We would meet sporadically about it; usually before meeting with [KB]
. We didn’t have scheduled meetings. But we met [KB]
regularly”. (4-C)
Two key themes emerged regarding the communication of staff with EIDM experts through the implementation of the intervention: the role of the KB and the librarian.
The KB was the main deliverer of the intervention and had a critical role to fill in all steps of the process at each of the public health units, as pointed out by the organizational leader: “Everybody in this organization sees the [KB] role as absolutely central for success. And every time I say, ‘Well, where I am going to get the money for this?’ I say. ‘I better find it because we are not losing it’.”(4-A), or by a manager at unit C: “it was very helpful to have someone to go to, to be able to help us with those steps along the way. So I think it was that part was certainly appreciated.” (1-C). In addition to personal competencies, the physical presence and accessibility of the KB was mentioned frequently as a reason for her popularity, as pointed out by a project specialist at unit A: Her desk is right opposite the office of MOH. So she is not buried. She is front and center. […] Anyone who walks by can see it. (2-A)
The KB was also widely seen as an external and neutral person, not involved in the policies and hierarchies of the department, as noted by a project specialist at unit A: “She is objective in the sense that she is not involved in the dynamics and politics in each division, so if you go to her for advice she can provide that without having those things in mind.”(5-A).
Another important professional supporter of EIDM through the implementation of the intervention was the librarian associated with each unit. Although the public health librarians were considered to be an integral part of EIDM process by the informants in all the three health units, the perceived level of involvement and usefulness of librarians differed considerably across sites. At unit A, during the study period the unit hired new librarians who were formally assigned to do rapid reviews and develop and update search strategies. Likewise at unit C the librarian was involved in the process and was helpful in assisting staff through EIDM steps. In contrast at unit B a recurring theme in interviews was that the library system did not help staff meet EIDM standards. The library basically provided
“a million single studies” and the staff struggled with classifying and appraising the information, as expressed by a consultant at unit B:
One of my biggest frustrations, … is [the study] was trying to work organizationally with [Unit B], and one of the greatest barriers is the way our library access is used… When you request a search by the library you get a stack of papers with no order, a mix of single studies and systematic reviews. You get a hodge-podge which for most of us… I wouldn’t have before known how to tease [out] what was what, how to quickly go through and see which one was synthesis and which one weren’t. It is a bit overwhelming. (2-B)
Relational outcomes
Especially at units A and C, completed reviews were presented in department-wide research events and other local meetings. Quantitative analysis [
20] showed that the information seeking networks evolved towards a more centralized structure over time, in which the staff who were already central at baseline, staff with higher baseline EIDM behavior scores, and larger improvement in their EIDM behavior scores gained even more centrality. Only at unit A highly engaged staff also shifted towards the center of information seeking networks. Interviewees mostly focused on the
recognition of participants in trainings:
Especially at units ‘A’ and ‘C’, trained staff had various opportunities to present their work to a larger audience both inside and outside of the health unit. As one manager noted, “Twice a year we have our research and knowledge exchange symposium, and so all the unit is there to hear about EIDM. And they see and hear from various sources who is knowledgeable on the topic” (1-C). Presenting work in those venues resulted in widespread recognition of trained staff by their peers, as expressed by a leader at unit ‘A’: “If your work has been showcased in that venue, people from all over would say, ‘Oh, you did a really good job on that’. They might not even know the name of that person before, and all of a sudden they know who they are.”(4-A)
Organizational leaders played a significant role in recognizing trained staff, as described by a manager at unit ‘A’: “Because there is kind of a little bit of mystique about [this] rapid review business … [the unit leader] makes a big fuss about it. When you get in the meeting with [the leader], there is a fair amount of social capital attached to joining the rapid review [team]. It’s a little bit like you are kind of famous!”(1-A) Recognition of the newly gained expertise of trained staff also occurred through word of mouth, as pointed out by a manager at unit ‘C’: “Here we are a smaller division; lots of people just knew [who were engaged in the intervention] by osmosis. Because we talk [with each other] at our managers’ meetings.”(4-C)
In contrast, in the larger and more diffuse public health unit ‘B’, word of mouth was not as frequently effective in promoting recognition of trained staff :
“The people who were involved were selected and were sent stuff electronically; [but]
that wasn’t in our newsletter or anything. So I don’t think they had any exposure.”(4-B)
An interesting and unanticipated consequence of unit A’s strategy to target project specialists and managers, and promote the individuals who were engaged in EIDM activities was a negative reaction of the staff who were not chosen to take part in the intervention. The selected staff enjoyed working in an
“ivory tower” environment of recognition and prestige. But many staff who were not chosen felt left behind, as indicated by a librarian at unit A
: “That mechanism of picking resulted in emotional responses for not being chosen… because they were not viewed as elite. They were not part of the club.”(3-A) [indirect experience], As another project specialist at unit A noted [indirect experience]:
I think the front line staff that were not been sent to [the university-affiliated one-week workshop], they felt left behind and frustrated, because it was like all these staff specialists are moving forward and advancing their skills, and they are gonna be used more and appreciated more by management, again this is the sense I got. That definitely caused tension, feeling of that ivory tower of the specialists. (5-A)
Paradoxically, being chosen for training resulted in a heavier workload and more pressure due to greater responsibilities. Prestige and workload were positively correlated, as explained by a project specialist: “but there is also more pressure on us too, so it goes both ways. For if you got more trained there is also more pressure on you to do more work” (5-A); or pointed out by a leader at unit A: “We happen to have created very hard work that seen as very desirable to do.”(4-A)
The informants at units B and C did not observe such reactions among staff. For example, when asked about the possibility of such consequences a public health nurse at unit C indicated: “it is not about the prestige” (3-C). The staff who were engaged only became more skilled to help others and not necessarily more popular or advantaged.