Background
It is well established that the creation of new knowledge through biomedical and health services research does not automatically lead to widespread implementation or health impacts [
1]. To maximize health system resources and improve patient outcomes, it is increasingly important to close the research-practice gap by ensuring that research knowledge translates into action – a process called knowledge translation (KT). KT is defined as the synthesis, exchange, and application of knowledge to improve the health of individuals, provide more effective health services and products, and strengthen health care systems [
1]. Current approaches to KT are largely focused on aligning the behaviours of health professionals with best research evidence; however, ever-increasing healthcare complexity and health professional time constraints are barriers to effective research use [
2,
3]. An emerging approach to KT is directing information to health consumers (i.e., patients, parents, caregivers) to increase their knowledge and participation in health decision-making.
In the field of child health, connecting parents and caregivers to research evidence has the power to improve health decision-making and reduce health system costs [
4]. Traditional approaches used by health providers to share information with parents and caregivers have been found lacking. For instance, verbal information is often brief [
5] and written information is often too complex for most adults to comprehend [
6,
7]. There is little guidance on the most effective approach, content, duration, and intensity of information provision for the diverse population that parents represent [
8‐
10].
While KT interventions encompass a wide array of strategies to bridge the research-practice gap, including individual, organizational, and structural interventions [
11], KT tools are a sub-group of KT interventions that present research-based information in user-friendly language and formats to provide explicit recommendations, and/or meet knowledge/information needs [
12]. KT tools are particularly suited for lay audiences, including parents and caregivers. It is hypothesized that KT tools may foster and empower ‘effective consumers’ with research knowledge to inform their health decision-making [
13].
The purpose of this scoping review was to identify previously published effectiveness research on child health-related KT tools for parents/caregivers. We sought to understand the breadth of KT tools that have been developed and evaluated (including their intended purpose), how they are being evaluated (including the outcomes selected), and whether they are demonstrating hypothesized effects. Understanding the evidence-base for KT tools for parents/caregivers in child health and identifying gaps in this emerging field is a critical next step to inform KT science for health consumers.
Methods
This scoping review was guided by the rigorous, systematic methods outlined by Arksey & O’Malley [
14].
Search strategy
A comprehensive literature search was designed and implemented by a health research librarian (TC) in eight databases: Medline, Medline In-Process & Other None-Indexed Citations, EBM Reviews, Embase, PsychINFO, CINAHL, SocINDEX, and Web of science. The search included language (English only) and date restrictions (2005 – June 2015) (search strategies and terms in Additional file
1). Date restrictions reflect the advent of KT science [
12,
15] and the emergence of KT targeting health consumers [
16].
Study inclusion criteria
The inclusion criteria are outlined in Table
1. In brief, we were interested in any primary research evaluating the effectiveness of a KT tool on a child health topic and targeting parents/lay caregivers. A KT tool was defined as a tangible, on-demand product presenting research-based information in user-friendly language and format(s) to provide explicit recommendations, and/or meet knowledge/information needs.
Table 1
Study inclusion criteria
1. Primary research study | Inclusive of all study designs. |
2. Evaluated effectiveness of an intervention | Defined as determining efficacy and/or effectiveness (i.e., does it work?). Studies examining functionality, feasibility, and/or acceptability to inform intervention development were excluded. |
3. Intervention evaluated was a KT tool | Defined as tangible (i.e., either material or electronic) products presenting research-based information in user-friendly language and format(s) to provide explicit recommendations, and/or meet knowledge/information needs. The KT tool must be available on-demand so that the target audience can mediate its use (i.e., when to use them, how often to use them, etc.). |
4. Intervention targeted parents/caregivers | Defined as individuals responsible for the health and wellbeing of child(ren) and are active-participants in child health decision making. |
5. Intervention provided research-based information on child health topics | Inclusive of all child health topics. |
Study selection
One reviewer (LA) conducted primary and secondary screening using pre-determined criteria (Additional file
2). A second, independent reviewer (XW) screened 10% of all studies to verify inclusion/exclusion decisions. Interrater agreement was determined to be ‘very good’ with a kappa statistic of 0.803 [
17].
Data collection
Data were extracted by one reviewer (LA). The following general variables were extracted: authors, year of publication, country, and journal of publication. Methodological elements were also extracted, including: study design, study focus (i.e., purpose), availability of a priori protocol, study population, sample size calculation, recruitment and retention, intervention and comparison groups, data collection methods, primary outcome(s) and measures. We also extracted the results for the primary outcomes, and author conclusions. Additional variables specific to the KT tools were extracted, including: child health topic, purpose of tool, description of tool, tool development approaches (e.g., including end-users, theoretical basis, and preliminary research conducted prior to effectiveness evaluation), type of tool, and number of interacting tool elements.
Methodological quality assessment
Scoping reviews do not typically include critical appraisal of individual studies [
14,
18]. This has been acknowledged as a limitation of the Arksey & O’Malley method [
19]. New methodological recommendations include methodological quality assessment to demonstrate gaps in the evidence-base and demonstrate feasibility of future systematic reviews [
19]. However, studies should not to be excluded based on these methodological quality ratings [
19], which is how we proceeded in this review.
For randomized controlled trials (RCTs), methodological quality was assessed by one reviewer (LA) using the Cochrane Risk of Bias Tool [
20]. This tool has been deemed the most comprehensive for assessing potential for bias in RCTs [
21] and has become the standard approach for systematic reviews [
22]. A global quality rating of low, high or unclear risk of bias is assigned to each RCT based on seven components: random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective reporting, ‘other’ sources of bias.
For all other quantitative study designs, methodological quality was assessed by one reviewer (LA) using the Quality Assessment Tool for Quantitative Studies [
23]. Content validity, construct validity, and inter-rater reliability have been established for this tool [
24]. A global quality rating of weak, moderate or strong is assigned to each study based on eight components: selection bias, study design, confounders, blinding, data collection methods, withdrawals and dropouts, intervention integrity, and analysis.
Data analysis
A descriptive analysis of the extracted variables was conducted. The WIDER Recommendations Checklist was applied to describe the reporting quality of the KT tools [
25] (Additional file
3). Since the studies assessed primary outcomes at different levels, a classification scheme of outcomes for assessing patient-focused interventions was applied [
26] (Additional file
4). Study results were described as positive effect, mixed effects, no effect, or unclear in relation to the intended impact on the primary outcome(s). A narrative summary of these effects was performed considering the nature of the intervention, topic, and study design features.
Discussion
This scoping review has demonstrated that several different KT tools have been specifically designed for parents/caregivers on diverse child health topics, which include a variety of single- and multi-component strategies. Few KT tools demonstrated positive effects for primary outcomes; the majority of studies showed mixed effects within and between primary outcome categories. Only two studies showed strictly positive effects and both evaluated multi-component KT tools. Three studies showed no effect and these evaluated single-component KT tools, specifically they were all non-electronic, written materials. This suggests that multi-component KT tools may be more effective for health consumers, specifically parents and caregivers. While we did not conduct formal comparisons, these findings contradict previous research indicating the effectiveness of patient-focused interventions decreases as the number of intervention components number increases [
45].
This review demonstrated that the most common design was the RCT (
n = 10), which is recognized as the most rigorous design for evaluating effectiveness [
46]. All included RCTs were assessed as unclear or high risk of bias; further, included non-RCT studies all had substantial methodological weaknesses. Given these methodological weaknesses, it might be expected that interventions would be more likely to demonstrate an effect, particularly for the primary outcomes; however, most studies did not demonstrate significantly positive results on the primary outcomes of interest. This raises three considerations: 1) was the design or certain design features (e.g., sample size, nature of the comparison) inappropriate or inadequate to assess effectiveness; 2) were appropriate outcomes selected and measured to accurately assess intended impact and establish effectiveness; or 3) have the KT tools been appropriately developed and incrementally assessed to establish effectiveness?
There were several design/methodological issues that may have impacted the effectiveness results. None of the included studies was shown to have low risk of bias (i.e., high or unclear risk of bias ratings only). Low methodological quality ratings were due to deficiencies in multiple categories in both tools. Interestingly, no a priori protocols were available and only 4 studies retrospectively registered their protocols. While a priori protocols may not yet be standard for all study designs, it is standard practice to register a priori protocols for RCTs [
47,
48]. Additionally, only 3 studies (16.7%) demonstrated adequate power to detect statistical significance of the primary outcomes. This information, generally provided in study protocols (if not also in primary publications), is a key aspect of effective comparative studies in health research [
49]. With strict journal length restrictions, it is difficult to determine if high risk of bias/weak methodological quality can be attributed to lack of reporting and/or poor study design and execution; however, the publication of study protocols has been proposed as an important approach in the primary prevention of poor medical/health research [
50], particularly selective outcome reporting [
51].
There is little agreement on the best outcomes and measures to determine effectiveness of patient-focused interventions [
26]. Applying the Coulter & Ellins patient-important outcomes framework was useful because it helped to reduce the ‘noise’ and classify multiple outcome measures within and across four distinct outcome categories. Studies used many different outcome measures across a variety of proximal (i.e., patients’ knowledge) and distal (i.e., health services utilization/health behavior) outcome categories. Just under half (44%) of studies skipped proximal outcomes (i.e., knowledge) and instead only measured more distal, behaviour-related outcomes. Two such studies showed statistically significant positive effects of the KT tools, but methodological quality concerns (i.e., high and unclear ratings) and no a priori protocol or sample size estimation limit our confidence in the link between the KT tool and these distal, health behaviour and health status changes. There may be other mitigating factors in the ‘black box’ between the interventions and outcomes. Additionally, the use of multiple outcome measures within the same primary outcome category and/or measuring multiple primary outcome categories most often resulted in mixed effectiveness (
n = 9). It is difficult to interpret these results without authors’ providing explicit rationale linking primary outcome(s) and measures to the intended effect of the KT tools. It is important to note that only four studies described the theoretical basis for the KT tool; more explicit theoretical underpinnings may help in tool development and linking tools to intended outcomes.
Unfortunately, the overall poor intervention reporting quality in the literature [
52‐
55], including development approaches/methods [
46], theoretical basis [
56,
57], and end-user involvement [
58], results in limited understanding of intervention components and relationship/interaction between these components, which are responsible for observed changed and desired effects on outcomes [
59]. These issues were exemplified in this review with poor reporting across all WIDER Recommendation categories. Without detailed understanding of these important elements, KT tool sustainability, replication, scale-up and future development efforts are limited [
25].
Effectiveness evaluation is typically resource intensive, yet we need to understand whether KT tools are effective for a lay audience prior to large-scale implementation. Formative research (i.e., qualitative, feasibility, pilot studies) prior to launching into effectiveness evaluation may be essential to attend to intervention development and implementation issues, refine effectiveness evaluation protocols, including most appropriate outcomes and measures, and ensure potential impact [
57,
60]. However, only half (
n = 9) of the included studies described or referenced preliminary research that was conducted to inform the current effectiveness evaluation study. Both studies in this review that demonstrated significant positive effects on primary outcomes referenced such preliminary research. Unfortunately, these studies did not provide sufficient detail to guide future KT tool evaluations; however, future research could attend to this need.
There is also a growing body of literature to support the use of qualitative research in the design and implementation of RCTs [
61‐
64]. Qualitative research has been used to add value to trials in the areas of bias, efficiency, ethics, implementation, interpretation, relevance, success, and validity [
65]. Novel study designs, beyond RCTs, possibly including mixed methods, may explain why the KT tools worked or not, help explain and interpret effectiveness results, and explore the implementation process [
57].
Strengths & limitations
This scoping review provides a detailed summary of the state of the science for the emerging field of KT tools for parents and caregivers on child health topics. By conducting critical appraisal using two rigorous frameworks for multiple study design types, this review offers important methodological advancement of the Arksey & O’Malley (2005) scoping review method [
14,
18,
19]. One limitation is the lack of a second reviewer to verify data extraction and critical appraisal, as would be expected in a systematic review. Another limitation was the lack of a classification scheme for the KT tools; the Coulter & Ellins patient-focused intervention classification was not used because it had a broader scope than desired [
26]. Multiple, overlapping frameworks are a persistent problem in the KT field [
66,
67]; however, the recently published AIMD meta-framework may be the solution [
68] and future research should explore KT tool development, reporting, and classification with this new framework.
Conclusions
KT tools offer a promising approach to communicate complex health information to health consumers. While a breadth of KT tools have been developed to provide research-based information on a wide variety of acute, chronic and public health/health promotion topics in child health, improved reporting is essential to ensure intervention design is appropriate for desired change and that well designed interventions are replicable. Additionally, increased methodological rigor is needed to determine the effectiveness of the KT tools. This includes the publication of a priori protocols, sample size calculations, primary outcome identification, and attending to multiple outcome measures and mixed results. More preliminary research, including KT tool development involving the target end-users and usability testing prior to large-scale trials, may be important to optimize KT tool effectiveness. Further, ensuring all necessary intervention and methodological components are attended to before and during effectiveness evaluation will help provide a more solid scientific base for KT targeting health consumers.