Background
The literature indicates that successful knowledge mobilisation partnerships have commonalities, including partners valuing different types of knowledge and contributions, participative processes in which the design, conduct, interpretation and implementation of research are negotiated and reciprocal, reflexivity and management of relationship dynamics, and an emphasis on building dialogue, trust, mutual respect and shared goals [21, 23‐25]; however, this is not easy to achieve [26, 27]. Mobilising knowledge is a messy, conditional and profoundly context-dependent process [14, 28, 29]. Every partnership will have unique features and, like all complex systems, will be in flux [21], contingent on myriad factors, including relationships, values, leadership styles, incentives, structural and financial supports, role allocation, the type of problem being tackled, and their social and political contexts [3, 27, 30, 31].“… knowledge mobilisation is not just about moving a clearly defined set of ideas, concepts, research techniques or information from here to there. Rather, it is about grappling with which forms of knowledge are apt in which contexts and how they can be strengthened through use” [22].
The Australian Prevention Partnership Centre
Aims
Methods
Methodology
The Partnership Centre programme model
Data collection
Method | Period | Objective | Data collection details | Analysis |
---|---|---|---|---|
1. Interviews with the Centre’s chief investigators and funding partners | January–March 2016 | To explore experiences of involvement, perceptions about the Centre’s functioning and achievements, and areas for improvement | Semi-structured interviews with chief investigators (n = 21/31, including researchers and policy-makers) and funding partners (n = 5/5) named on the original grant; a total of 26 participants | |
2. Interviews with members of the Centre’s research network | July–August 2017 | Semi-structured interviews with a representative purposive sample (selected by role and career stage) of PhD students, research officers/fellows and project leads, i.e. people involved in Centre research but not named on the original grant (n = 19); this was approximately 1/3 of the research network at that time | ||
3. Interviews with policy partners | June–July 2018 | Semi-structured interviews with policy-makers (n = 18) who self-nominated for follow-up having completed a brief online survey about engagement with the Centre; the survey was advertised on the Centre website and via the Centre newsletter; one policy-maker was excluded from interviews because they had recently taken a paid role with the Centre All interviewees gave informed consent; interviews were audio recorded, professionally transcribed and then checked for errors by the interviewers | ||
4. Partnership survey (a cross-sectional anonymous online survey) | June 2015 October 2016 August 2018 | To explore the Centre’s functioning according to partners (policy-makers, practitioners, researchers) and Centre staff; the survey covers perceptions of leadership, governance, resource allocation, collaboration and engagement | All Centre partners were invited to participate via personal email; survey hyperlinks were included in Centre e-newsletters and on its website; survey statements relating to aspects of the partnership were scored on a 7-point Likert scale from ‘strongly disagree’ to ‘strongly agree’; participants were also asked to rate specified experiences of partnership and comment on what worked well and what might be improved; the baseline survey was completed by 50 people, follow-up 1 was completed by 97 and follow-up 2 by 59 people | Statistical analysis of closed questions by wave of survey and thematic categorisation of open-ended questions; further details about the analysis of survey data are provided in Additional file 4 |
5. Participant feedback on ‘systems thinking’ workshops | Routinely collected after each event since February 2017 | To elicit participants’ views of the functioning and value of events | Structured anonymous feedback forms completed by event attendees, including Centre partners and any other stakeholders who attended (n = 173 of approximately 230 attendees) | Descriptive statistical analysis and thematic categorisation of open-ended questions |
6. Routine process data about Centre activities, funding and growth | Continual | To record Centre inputs, reach and outputs, including how strategies are being implemented and any impacts | Collation of data from project reports, communication products/website access data, project outputs, meeting minutes, the Centre’s partner database, ‘feedback register’ and key performance indicators | Thematic categorisation of text data and descriptive analysis of quantitative data |
Data analysis
Results
Knowledge mobilisation strategies | Key governance and implementation strategies | Strengths and achievements | Stakeholders’ perceptions of benefits | Challenges and potential areas for improvement |
---|---|---|---|---|
1. Partnerships | • Involve partners in planning and governance • Require partners to commit resources so they have ‘skin in the game’ • Leverage existing cross-sector relationships to establish project teams, reach potential partners and create a networked platform • Connect with new partners and support current relationships | • Considerable growth in investigator team and partner organisations • Increased funding and resources from partners and government • Perception that skills are used effectively in the partnership and that the Centre’s benefits outweigh its costs | • Most interviewed policy-makers and funders regard the Centre’s work as useful, innovative and important • Policy-makers valued opportunities to shape research, access resources and forge connections within a collaborative network • Researchers valued linkage with (and more likely impact on) policy | • Partnership governance could be more transparent • Greater awareness of conflict resolution options needed • Some policy-makers found it hard to attend forums or to be ‘heard’ at them • Some uncertainty across stakeholders about how to tap into the Centre’s network |
2. Engagement | • Funding teams of researchers, policy-makers and practitioners to work together • Interactive and networking forums for researchers, policy-makers and funders • Strategic communications, e.g. website, newsletters, narrative reports, policy/practice-friendly research summaries • Co-ordination and administrative support to link projects, manage funding and partnership agreements, and act as contacts for queries | • Partners see value in committing their time to the Centre and believe their abilities are being used effectively • Partners are getting the information needed to stay abreast of developments and opportunities, and to contribute meaningfully to the Centre • Most partners feel the Centre has a clear vision | • Access to high quality resources that are relevant and applicable to policy work • Awareness of Centre developments and opportunities • Engagement with systems science and other innovations • Access to online networked events and practice groups, and mentoring by Centre staff | • It has been hard to create a shared vision for all partners • Stakeholders can struggle to identify relevant projects or get involved in projects • Geographic distance from metropolitan areas and the coordination hub is a barrier • Belief that the partnership is achieving more than partners could do alone has decreased |
3. Capacity and skills | • Dedicated capacity-building staff develop resources, run events and provide mentoring • Expert-run workshops and webinars • Cross-project forums and networks, including a community of practice in applied systems thinking • Investment in early-career researcher development (scholarships, postdoctoral fellowships and funding to attend conferences) • Cross-sector placements | • Capacity-building activities are frequent, varied, well-attended and well-received (e.g. perceived as useful and a good use of participants’ time) • High levels of reported satisfaction with the Centre’s communications, resources and capacity-building activities | • Access to national and international experts • Development and application of new knowledge and skills, e.g. in ‘real word’ research methods and systems approaches • Better understanding of the research-policy interface • Access to educational resources | • Cross-sector placements are hard to secure, often due to incompatible organisational requirements |
4. Co-production | • Encourage cross-sector investigator project teams • Shape projects and collaborative opportunities around partners’ developing agendas • Host roundtable events and exchanges between researchers, policy-makers and practitioners to foster collective work and debate | • Multiple projects are engaged in cross-sector co-production • Many policy-makers are involved with different levels of seniority participating in different ways • Most policy-makers report examples of genuine co-production in which they saw themselves as full partners • Partners identify innovations arising from co-production | • Co-production allows partners to shape project directions (especially via shared priority-setting), gain access to expertise and resources, increase mutual learning and share ideas • Dramatically improved research relevance • Translation of research to policy is ‘built-in’ • Involvement in priority-setting justifies policy-makers’ time commitments | • Projects are less attuned to the needs of non-funding policy-makers as they are less involved in co-production • Different views of co-production: is it shared decision-making or generating research questions collectively or co-conducting research? • Greater facilitation of shared decision-making and problem-solving may be warranted • Co-production challenged by personalities, competing time frames and its own logistics |
5. Knowledge integration | • Discussion forums to create linkages and synergies across current and future projects • Resourcing for high quality strategic evidence synthesis and communication • Dedicated roles and tasks regarding forging project connections, synthesising research findings and sharing knowledge | • To some extent, discussion forums are facilitating linkage and information-sharing | • In some cases, there are synergies across multiple projects | • More work is needed to create linkage, consolidate findings from separate projects and forge a coherent prevention narrative |
6. Adaptive learning and improvement | • Evaluation: surveys, social network analyses, stakeholder interviews, process measures, key performance indicators and events feedback • Collate formal and incidental feedback in a register • Distribute evaluation results and discuss in Centre forums to ‘close the loop’ and enable action • Build reflection into the Centre’s quarterly reporting procedures | • There is some evidence of the Centre’s adaptivity and increasing flexibility | • In some cases, a dynamic and policy-responsive work plan | • More use could be made of evaluation information • Greater transparency at the executive level could help partners to see what information is considered and how it is acted on |
Partnerships
Partnership survey data (Table 3) shows that perceptions of partnership governance strengthened in the first few years of the Centre, but scores dropped below 15-month follow-up levels at the 3-year follow-up point (this was a significant drop for items 7, 8 and 10). Management of conflict (item 9) had the lowest agreement at baseline but, at all timepoints, the majority of respondents fell into the ‘neutral’ category for this item, suggesting that they may have been unaware of a process for conflict resolution, possibly because they have not needed it. However, the data also indicated a perception that skills are used effectively within the partnership (item 3). Importantly, there was growth in agreement from baseline to the 3-year follow-up that the benefits of the Centre outweigh its costs (item 5).“In terms of in-kind [contributions] we’ve tripled what we thought would be given in that way because of the interest and the relevance of the work, so that’s something that’s been really important … The whole idea-sharing and the embedding research with embedded capacity-building has been a crucial component, and very positive.”
Categories and statements in the partnership survey | Percentage agreement with survey statements | ||
---|---|---|---|
Baseline | 15-month follow-up | 3-year follow-up | |
Resource allocation | |||
1. Adequate financial resources are available | 73.9 | 65.6 | 68.6 |
2. Necessary skills are available in the partnership | 69.6 | 80.0 | 80.4 |
3. Available skills are used effectively | 39.1 | 62.9 | 62.8 |
4. Adequate partner time is allocated | 30.4 | 47.7 | 41.2 |
5. The benefits of allocating resources to the Centre outweigh the costs for my area | 47.8 | 44.8 | 58.0 |
Governance | |||
6. There are defined roles and responsibilities | 55.1 | 66.3 | 52.8 |
7. There is a clear process for planning and implementing activities | 53.1 | 58.7 | 44.2 |
8. There is a clear process for shared decision-making | 32.7 | 42.4 | 30.8 |
9. There is an effective process for managing conflict | 10.2 | 29.7 | 9.6 |
10. There is a clear framework for monitoring progress | 46.9 | 71.7 | 41.5 |
Leadership | |||
11. There is a clear vision for the Centre | 46.0 | 63.9 | 64.4 |
12. There is clear communication of the goals of the Centre to staff | 52.0 | 63.9 | 63.8 |
13. There is enthusiasm for achieving the Centre’s goals | 68.0 | 82.5 | 81.4 |
14. There are strategies for relationship building among partners | 54.0 | 74.2 | 61.0 |
15. There is strategic leadership for the Centre | 60.0 | 81.3 | 76.3 |
Engagement | |||
16. I understand what the Centre is trying to achieve | 71.1 | 79.6 | 67.9 |
17. I see value in committing my time to the Centre | 84.4 | 82.8 | 75.5 |
18. I understand my role and responsibilities within the Centre | 71.1 | 72.0 | 66.0 |
19. My abilities are used effectively in the Centre | 40.0 | 57.0 | 54.7 |
20. I receive the information I need to contribute meaningfully to the Centre | 48.9 | 66.7 | 58.5 |
21. I feel respected and valued as a member of the partnership | 64.4 | 77.4 | 64.2 |
22. I believe the Centre partners are achieving more together than they could alone | 55.6 | 81.7 | 69.8 |
Collaboration | |||
23. There is trust and respect among partners | 65.9 | 79.3 | 76.9 |
24. There is sharing of ideas, resources and skills among partners | 52.3 | 77.2 | 61.5 |
25. There is collaboration to solve problems | 45.5 | 69.6 | 49.0 |
26. There is effective communication among partners | 38.6 | 59.8 | 50.0 |
27. There are new and strengthened working relationships among partners | 59.1 | 74.7 | 64.7 |
Many interviewees talked about the Centre’s network of research and policy experts as its major asset; however, a number of participants, especially more junior policy-makers and practitioners in non-funding agencies, indicated a desire for stronger ties to the network and expressed uncertainty about how to tap into it. This was echoed by some researchers who wanted greater cross-project collaboration but felt that being employed on discrete projects led by separate chief investigators created limited opportunities. Some were unsure if cross-project collaboration was permitted given they were paid from specific project budgets.“… there were a couple of projects being discussed and the person I was sitting next to and I were going, ‘Hmm, I don't know how useful that's going to be’. But it’s not in the culture to say it in that kind of environment.”
Engagement
“… the actual communication of the decisions that are made and the processes could be clearer … how funding decisions are made, how priorities are decided on, what is the process if you've got an idea … [do you] first talk with decision-makers or do you need to talk to the management committee about it first?” (Researcher)
“In terms of the strengths of the [Prevention Centre] model, one has been maintaining communication with everyone. The newsletter that [the Prevention Centre] sends out is probably the only that I read ever because it is quite punchy, very useful and they only put points in there that would be relevant.” (Researcher)
“I often reach for things that the Prevention Centre has on their website … because they’ve been translated in very accessible language and also in a way that if we’re pitching something to the Minister, it says ‘Well what do we know?’ The ‘so what?’ is really helpful when you’re writing up something rapidly, to be able to see ‘Okay, what’s the practical implication of that?’ I find that very useful.” (Policy-maker)
There appeared to be some patterns in engagement. Interviewees who reported having a clearly defined role within the Centre, especially if they received project funding, tended to feel more engaged; conversely, those who reported limited engagement often commented on a lack of projects that aligned with their area of expertise, or an unclear understanding of how they could be involved. Location and frequency of contact also seemed to make a difference; distance from major metropolitan areas (especially from Sydney, where the Centre’s coordinating team are based) and lack of regular interactions damped engagement, but participation in online networked events and practice groups, and mentoring by Centre staff, seemed to boost identification with the Centre and enthusiasm about its work.“… we’re hopeful that’s going to bring about some change, but I think it’s already started bringing about some change in our own team … Whether or not we can then broaden that out to have an impact on others outside of the population health and planning area, remains to be seen. That’s definitely a work in progress but, certainly, the foundations are all there.” (Policy-maker)
Capacity and skills
“[The Centre] is big on bringing internationally renowned speakers on systems thinking and other related areas to Australia. I always make a real effort to attend those because they're always such high quality. I'll research the speaker and look at their work and that leads you onto another body of literature that you hadn't considered.” (Researcher)
“By working with different professionals and people with different backgrounds, by default you’re building your capacity to understand the different domains and what their priorities are and how you might go about communicating something to one group vs another.”
The investment in early career researchers was seen as key to developing the next generation of prevention researchers. However, despite a commitment to find placements for researchers in policy or practice agencies, and vice versa, these had been hard to secure, largely due to incompatible organisational requirements. Government agencies and research institutes tend to have security and access restrictions that would have excluded seconded staff from workplaces, data and/or technology. As an alternative, the Prevention Centre hosted research placements enabling policy staff and researchers to work jointly on projects. Feedback from these participants was overwhelmingly positive thanks to the level of support and opportunities provided.“It’s information that is important, but it’s new information or it’s presented in a different way and I think this is part of that – the messages that we’ve had around for a long time aren’t quite working so we need new approaches, new research, new ways of telling the story. And I think that that’s what we’re really getting out of the Centre.”
“The system dynamics modelling for childhood obesity has been a real success for us. It’s been incredibly beneficial for our programme planning and how we talk about the work that we’re doing, and what we expect the impacts to be.”
The partnership survey asked respondents to rate their level of satisfaction with the Centre’s communications, online resources and capacity-building activities. Satisfaction levels had risen over the years, with between 73% and 92% saying, at the last survey, that they were satisfied/very satisfied with these functions (data not shown).“… some of it is around access to new knowledge and methods, particularly around systems thinking and approaches to improving prevention policy and practise, so that’s both an opportunity to learn in a structured way but also mentoring around some of those approaches.”
Co-production
However, this meant that some policy-makers outside of funding agencies who had less scope to co-produce research experienced the Centre’s outputs as less attuned to their needs.“It’s the participation which gets you thinking about your practice, gets you thinking about different [policy] opportunities. Because we’re involved in this work we don’t wait for some glossy two-pager because we co-created it, and we’ve communicated about it as we go.” (Policy-maker)
“[We are] working collaboratively in co-production both with practitioners and researchers. That is often a motherhood statement … but I think the results do speak for themselves with the partnership in the Prevention Centre; that it really does work collaboratively and co-produce work.”
Chief investigators, who had oversight of project teams, were particularly keen to advance a collaborative culture. They noted that co-production depended on the skills and experience of individual partners, bolstered by existing relationships, so that co-production was a familiar model for some teams but a cultural challenge in others. Researchers generally agreed that it was not possible to engage equally with all partners – personalities mattered (some policy-makers were less approachable or responsive) as did the quality of existing relationships.“[The Centre] goes through a prioritisation process with an executive which includes policy-makers which, I think, legitimises that and allows us — my colleagues and my staff — to legitimately work on projects and take a bit of time out to focus on it … because we’ve been actively involved in a co-creation process … and it’s doing work set through a priority-setting process — so, yeah, it’s more legitimate than other pieces of work.”
“We’re going to lengths to work with people. We’re not making major decisions without the policy-makers there. We’re walking around finding out what research questions they are interested in, we’re feeding that back. We’re figuring things out as we go along which … means my projects are taking longer and I’m a bit of a pest, but it does take longer when you do it that way … it’s a pain in the arse because these are very senior policy-makers … [and] you literally can’t get them at two or three weeks’ notice. You have to get into their diaries a month in advance. So, getting all of the right people into the right room to have a meeting that reflects the equity of decision-making that you want takes time.”
“… true co-production is where the researchers and decision-makers sit together and come up with the research questions themselves. Whereas the way [the Centre] seems to work is the decision-makers have a question that needs to be answered or researchers think that there is something interesting and you take it to each other to see if you can actually make [it fit] … I don’t think there’s really ‘true co-production’ in terms of both researchers and decision-makers coming up with the questions collaboratively.”
“I can understand how it is that researchers end up doing most of the work in partnership research, they end up being the researcher, when in fact it is meant to be a little bit more equitable in the sharing of that role.”
“… they [researchers] are doing the heavy lifting of the analysis and doing a lot of the grunt work that we otherwise wouldn’t be able to do internally … So the strength to the Centre is that you’ve got someone there as a dedicated resource to focus on those issues.”
These concerns were echoed by findings from the partnership survey (Table 3), which showed initial growth in partners’ positive experiences of collaboration (items 25 and 26 were significantly higher at the first follow-up) followed by a statistically significant decline, although all other items remained comparable to baseline. Less than half the respondents in the most recent survey agreed with the statement that ‘There is collaboration to solve problems’ in the Centre. These results may reflect the uneven process of collaboration seen in the qualitative data above but also, in the case of the Centre, the disorientation and uncertainty associated with the loss of a pivotal team member and coming to the end of a funding cycle.“I don't feel as though I am [involved in priority-setting]. No. I feel as though there’s a smorgasbord of opportunities that I can and cannot get involved in … So you nominate the ones that you have an interest in. I don’t think, others might disagree, that we had the opportunity to look at it as a whole and ask ‘Well, how does that contribute?’”
Knowledge integration
“I was involved in some of the economic evaluation stuff. There were independent projects not really talking to each other, but [a member of the leadership executive] came in and then chaired all of them and ran a single meeting … that was fantastic because … there was a lot of overlap and then the projects were consolidated.”
“The thing that bothers me more is that I think it might look very scattered. We gave some money to this person and they did this and that was good and it got published. Somebody else did something else over here. I’m worried that it’s going to lack some kind of central organising theme that [would make] the Centre more than the sum of its parts.”
Adaptive learning and improvement
“I think it’s dynamic, and that’s great. I think it needs to be and it’s clearly being very responsive to emerging issues and to new ideas.”
“I think it’s been very mixed. I think we’ve helped develop the research questions but mostly the researchers are the ones who come forward with the questions. We've then had drafts to comment on but by the time you get draft as a proposal... It’s a bit late. You haven’t sat and brainstormed the research questions first together.”
This issue is also well established in the literature [25, 46] and, arguably, should be an axiomatic concern for a partnership committed to co-production. Yet, in the 2018 interviews with policy-makers, lack of shared decision-making was still a concern. Although some policy funders reported close involvement in specific key decisions, this was inconsistent, with limited capacity to set the direction of research programmes or, in some projects, even to be consulted in setting research questions or in developing the research plan,“If someone was working in a true co-production mode, how would you know? What would you measure? In my mind, what you would measure is something around the integrated decision-making capacity. To what extent is there a shared responsibility for the project? Where are the decisions being made about what are the next steps and how things will be done, and what the data means? I actually don’t see a lot of that happening in the Centre.”
“… It was very tokenistic, and it wasn’t good because, essentially, at a planning session we were told that we were going to be involved in a project and then I hadn’t had any conversations with the person about the project. The first I heard about it was, I was asked to look at an ethics approval for the programme of research. And no one had spoken to us about any of the research. No one.”