Background
It has long been suggested that partnerships between those who produce research and those who use it are likely to enhance the relevance of research and facilitate its use [
1,
2]. A variety of terms have been used to label this concept, each are subtly unique, and none are viewed as the over-arching or gold-standard term [
3]. In the health sector in Canada and elsewhere, this co-production of knowledge is commonly referred to as integrated knowledge translation (IKT) and defined as an ongoing relationship between researchers and decision-makers (clinicians, managers, policy-makers, etc.) for the purpose of engaging in a mutually beneficial research project or program of research to support decision-making [
4]. IKT is viewed as an approach or set of processes that can lead to the generation of knowledge for optimizing health care delivery systems and improving health system performance and associated outcomes [
5]. Decision-making research in health care settings shows that complex problems require complex solutions involving input from individuals with different expertise and perspectives and iterative, generative processes to formulate, execute, and evaluate solutions [
6]. Collaborative knowledge generation, as promoted through IKT approaches, involves ongoing, dynamic interactions among researchers and decision-makers, and represents an ideal means by which to address complex health care problems [
7].
Empirical research in health care settings has demonstrated the concrete benefits of IKT. For example, in-person contact with researchers has been repeatedly cited by decision-makers as the most influential factor determining their use of research evidence [
8,
9]. Interviews with participants of nine researcher-decision-maker partnerships funded by the United Kingdom National Health Service revealed that all achieved improved clinical care through a variety of IKT approaches [
10]. An exploratory study of a partnership among university administrators and Scottish health authority social workers revealed several impacts including enhanced dialogue among partners about priority health issues, and incorporation of research results into a training curriculum for social workers. As a result, social workers reported the use of research in formal health authority processes and enhanced skill and confidence in using research in their practice [
11].
IKT appears to improve the uptake of research into policy and practice through a variety of mechanisms. Collaboration between researchers and decision-makers may reveal differing perspectives, expectations, and values, leading to greater understanding and improved communication, which creates trust and a shared vision that enable more effective and sustained partnership, thereby contributing to the capacity for IKT [
12]. On a practical level, decision-makers can inform research questions that are relevant to practice or policy; refine research methods and/or data analysis; interpret findings based on their contextual knowledge; and disseminate or implement findings or products [
13]. Decision-makers benefit from interaction with researchers through a broadened reflection on their own activities, enhanced knowledge and skills, information about other pertinent research, and new contacts with other researchers or decision-makers [
14]. Researchers benefit as they gain a nuanced understanding of the policy or practice environment, develop and pursue research questions that have real-world applicability, and, through ongoing conversations with decision-makers, interpret results with a deeper understanding of contextual circumstances which, in turn, enhances the usefulness of the research findings.
Despite the emerging evidence of IKT’s positive impact, IKT is not yet widely practiced or well understood. Research directors in Canada reported that researchers tend to use traditional means of conducting and disseminating research rather than IKT approaches [
15,
16]. Similarly, a recent survey of health policy experts active in 30 European countries revealed little use of IKT other than isolated instances of embedded researchers in government research institutes or on advisory committees [
17]. Lack of engagement in IKT may reflect an inability to overcome the challenges inherent in coordinating complex, protracted initiatives with multiple stakeholders holding different views and pursuing different interests, or it may reflect a lack of incentives for researchers and decision-makers to engage in the more protracted and, hence, costly processes of knowledge co-generation [
18,
19]. Reflection on the challenges faced by researcher-decision-maker partners that investigated primary care networks resulted in several recommendations to facilitate IKT including the following: identify partners with pre-established links to ease and expedite interaction; establish clear expectations about role, scope, and contribution to foster trust and avoid role confusion and misconceptions; put in place mechanisms that initiate and support dialogue among partners; and jointly assess progress and implement changes as needed [
20]. A case study based on three health service delivery programs found that IKT activities were dynamic and not linear and highly influenced by the complex context within which decisions were being made including social and political norms [
21]. When IKT was formally incentivized with considerable funding through national initiatives, the number of interactions and projects increased but the research process was characterized as largely investigator-driven, and there was limited impact on health service delivery and outcomes [
22,
23].
Clinicians, researchers, and research funders have emphasized the need to understand how to foster and achieve IKT in the health sector [
24‐
28]. The purpose of this study was to characterize the nature of research in this area, describe IKT strategies that were empirically evaluated, reveal whether sufficient research is available to undertake a systematic review of the effectiveness of various IKT approaches, and also identify knowledge gaps for future IKT research.
Discussion
This scoping review was conducted to describe the knowledge base underlying IKT, gleaned from studies that described and evaluated IKT strategies, and identify gaps to inform future research. Thirteen studies were eligible. The most common form of interaction was meetings, but they varied in nature, aims, and frequency. All studies reported both barriers and enablers. While most studies achieved one or more positive outcomes, studies reported a wide range of positive and less positive outcomes. Given incomplete and inconsistent reporting of study design, IKT strategies, and outcomes, it was not possible to identify relationships between outcomes and contextual factors related to initiator of the partnership, dedicated funding, partnership maturity, nature of decision-maker involvement, presence or absence of enablers or barriers, or the number of different IKT activities employed in a given initiative. A number of studies assessed partnership formation. Given that the partnerships evaluated were at least 2 years old, it may not be reasonable to evaluate the influence of research on decision-making until more immediate outcomes such as learning about research, awareness and acceptance of research, mutual understanding, development of trust and goodwill, and an appreciation for the collaborative process are established. Another scoping review of stakeholder involvement in rehabilitation research found that stakeholder preparation was needed to understand research and fulfill their role [
33]. This took the form of formal and informal training and, in some studies included in that review, decision-makers were paid to participate in the training.
To the best of our knowledge, this study is among the first to attempt to identify the characteristics of IKT strategies and their potential association with outcomes using a rigorous approach. Our scoping review is distinguished from that of Jagosh et al. who published a realist systematic review on the effectiveness of community-based participatory research partnerships that included 276 studies [
36]. In participatory, action or community-based research, the intent is to improve the quality of service delivery, health equity, or clinical outcomes where community-identified rather than research-based solutions are emphasized, or researchers function as consultants. We chose to define IKT as partnerships between researchers and organizational or system-level decision-makers including clinician managers, health facility managers, and policy-makers for the purpose of academic research, although those partnerships may have enabled improvements in service delivery or clinical outcomes. IKT decision-maker partners are distinct in that they are specifically selected for their scope of responsibility and, hence, authority to invoke practice or policy change. This scoping review, which goes beyond anecdotal accounts, may serve as a springboard to the conduct of future research that specifically examines researcher-decision-maker partnerships.
Despite suggestions that lack of funding is a deterrent to practicing or achieving IKT [
3,
18,
19], in this study, formal IKT partnerships that were specifically initiated and funded by governments did not appear to eliminate barriers or report better outcomes compared with other studies that lacked such infrastructure. This included four studies evaluating Collaborations for Leadership in Applied Health Research and Care (CLAHRC) in the UK [
41,
43,
45,
46] and one study evaluating Academic Collaborative Centres (ACC) for Public Health in the Netherlands [
42]. Both of these national-level initiatives involved large-scale investment to foster IKT. Since partnerships may develop over time, and additional evaluations may be forthcoming, longitudinal evaluation of these important initiatives is warranted to identify beneficial outcomes. Another way to interpret these findings is that other barriers, enablers, or contextual conditions may be more important than funding to the formation and outcomes of IKT partnerships. Such insight was not afforded by this study because enablers, barriers, and outcomes were variable across studies and not consistently recorded or described. In future research, longitudinal analytic approaches may be useful to evaluate IKT impact and clarify the relationship between IKT approaches and outcomes. A time series design, for example, could be used to systematically track the evolution of partnership formation and better pin-point the activities or strategies that move the partnership from the formation stage into a more functional and active stage.
IKT was poorly and inconsistently described, evaluated, and reported in most studies, making it challenging to identify strong thematic areas. However, three important knowledge gaps were clearly identified. First, some studies evaluated the IKT initiative but did not describe or detail the IKT activities. Future researchers are encouraged, therefore, to capture and report the full extent of IKT activities, including the nature or mode of interactive activities (i.e., brainstorming sessions, data interpretation sessions, passive dissemination through websites), who is involved in which activity, who is leading the activity, and how often activities take place. This cumulative understanding will allow a nuanced typology of different IKT models to emerge. Those who plan and implement and/or evaluate IKT initiatives might employ the WIDER reporting checklist when they design such initiatives or report evaluative findings [
34]. The WIDER checklist recommends describing: the intervention (approaches, strategies), mode of delivery (intensity, duration, timing), intervention content (knowledge generated or shared), participants and their role (the characteristics of those sponsoring, delivering, and receiving the intervention), setting, and adherence or fidelity.
The second knowledge gap to emerge was the lack of explicit description of underlying theory or logic upon which IKT approaches and associated activities were selected and/or evaluated. As the WIDER checklist specifies, details are needed about how the IKT intervention was developed, change techniques used in the intervention, and the causal processes targeted by the change techniques to achieve particular outcomes. Therefore, future research could focus on identifying, describing, and testing relevant theory by which to design and/or evaluate IKT initiatives. First, it may be useful to conduct an interpretive synthesis of the findings reported here by analyzing enablers and barriers according to the context and design of IKT initiatives in the included studies. A scoping review is an appropriate starting point to understand the nature of the empirical work in the domain and to determine if a systematic review is warranted. Thus, we deliberately maintained a wide focus rather than targeting certain aspects of the IKT process. Our findings suggest that the empirical work in the area is just emerging, and thus it is premature to embark on a systematic review with a tight focus.
The third knowledge gap pertains to decision-maker involvement. IKT activities most often consisted of meetings between researchers and decision-makers. However, the nature of those meetings and the level of engagement of decision-makers in research-related decisions or research activities were not reported. In some cases, decision-makers were reported as playing a role in disseminating or implementing the results. It was largely not reported if decision-makers took part in any way in the conduct of the research or interpretation of the findings. Given that the nature of decision-maker involvement was largely under-described, we cannot say if the involvement of decision-makers throughout the course of a research initiative, which is the purported ideal [
3], actually achieves better outcomes. Other studies of IKT also reported that research remained largely investigator-driven [
22,
23], and decision-makers were often not directly involved as integral partners [
14]. Future research must examine a range of IKT approaches to identify the ideal timing and manner in which decision-makers must be involved for effective research uptake.
Several issues may limit the interpretation and use of these findings. The relatively small number of eligible studies may have precluded identifying with greater certainty the characteristics and contextual conditions required to foster and achieve IKT. This may, in part, be due to the fact that studies about IKT are difficult to identify. Other researchers have noted that the IKT literature is not consistently indexed in databases of published research [
33,
34]. Screening of search results was challenging due to the large number of search results to assess and limited detail in the studies by which to ascertain eligibility. This means that the resulting summary of IKT (Fig.
2) is inclusive of numerous characteristics and conditions that require further evaluation of their association with outcomes. Although we searched standard indexed sources of published medical literature, the search strategy may not have identified all relevant studies. Study retrieval was limited to journals that are indexed in the three databases that were searched. We did not search the grey literature, assuming that most empirical research on IKT interventions would be found in indexed databases. Many studies did not provide a full description of the research methods used or fully describe the research findings for all components of a case study or mixed methods research. Most studies collected qualitative data; however, they were often not complete or sufficiently detailed to extract clear findings with respect to enablers, barriers, and, in particular, outcomes. Given limited detail about IKT activities, it was difficult to chart data; however, we employed a rigorous methodology that complied with standard approaches for scoping reviews [
29,
30], and data were charted independently by two investigators for all articles to enhance reliability. In the absence of a universally accepted taxonomy with which to refer to IKT approaches, activities, processes, etc., it was challenging to describe and summarize how IKT was operationalized in included studies. Most studies reported one or more positive outcomes which may represent a bias toward reporting favorable findings; this is further underscored by the small number of eligible studies. Insufficient knowledge emerged from this scoping review to enable a full understanding of the variety of ways to promote IKT partnerships and engage in interactions. Therefore, we are unable to issue clear or justified recommendations in this regard. As more research on IKT emerges, this may become possible at some point in the future. Finally, some scholars [
30] have suggested that stakeholders ought to be consulted to validate and extend the interpretation of scoping review findings. Given the variation in results across the 13 articles, we suggest this step is better served in the future when additional research is available.
Conclusions
This scoping review found that most of the IKT initiatives that were evaluated achieved one or more positive outcomes. However, few studies were eligible, and IKT activities were poorly described, evaluated, and reported. Outcomes did not appear to be associated with initiator of the partnership, dedicated funding, partnership maturity, nature of decision-maker involvement, presence or absence of enablers or barriers, or the number of different IKT activities. Based on these findings, we cannot identify thematic areas across the studies to recommend particular IKT strategies or ideal contextual conditions. However, we generated a summary of the characteristics of IKT that have been examined and identified additional factors that remain to be examined. The findings can serve as the basis for future reviews, and for planning ongoing research that more systematically designs, implements, and evaluates IKT activities, and reports the findings with sufficient detail to reveal how IKT was associated with outcomes. Three important knowledge gaps were identified that lay the foundation for a research agenda in the area of IKT research.
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
ARG and WBB conceived the study, coordinated all aspects of its conduct, and prepared this manuscript. All authors assisted with study planning, data collection and interpretation, and drafting the manuscript. All authors read and approved the final manuscript.