Background
Awareness of the need for scientific approaches to dissemination and implementation (D&I), to provide for more effective and timely ways to translate research in practical applications, has increased over the past two decades [
1,
2]. With much of the cancer burden being preventable [
3], the need to close the estimated 17-year gap between publication of new research findings and their application to prevention in practice has become urgent [
4‐
6]. This increased awareness, and the development of D&I science has initiated programs training researchers in this domain [
1,
7‐
9]. Funding agencies are designating funds for D&I specific proposals [
10,
11], and there are increasing numbers of faculty positions focusing on D&I research [
12]; however, the availability of training programs has lagged behind the demand from a growing D&I research workforce [
13,
14]. Moreover, beyond efforts to identify requisite competencies, curricula, and desired outcomes of trainings, competencies must be put into action and tested [
4‐
6,
15,
16].
The Mentored Training for Dissemination and Implementation Research in Cancer (MT-DIRC) program is perhaps the first to incorporate systematic mentored training for D&I research with a focus on cancer control. It builds on experiences with D&I science training programs of varying depth and formats offered by several organizations in the USA and globally [
1,
17], with a particular focus on providing extended mentoring. Programs specific to mentoring in D&I science include the Implementation Research Institute (IRI) [
8] sponsored by the National Institute of Mental Health (NIMH), and the Training Institute for Dissemination and Implementation Research in Health (TIDIRH), sponsored by the National Institutes of Health [
7,
18]. IRI, which pioneered the approach of providing one-on-one 2-year mentoring, focuses on mental health researchers. The IRI recently added a component of training fellows to become mentors. TIDIRH, initially started as a 5-day training institute [
7], has evolved into an online-based program with a 2-day in-person session. Both IRI and TIDIRH have completed 7 years of summer trainings. The TIDIRH program has an informal mentoring component, but there is no established expectation for those mentoring relationships to last beyond the few in-person sessions. Outside the USA, Knowledge Translation Canada (or more informally known as KT Canada) has also offered summer training institutes that rotate cities and hosts throughout the years [
14,
19], and the Cochrane Collaboration, in conjunction with Public Health Insight at the University of Melbourne, offers a 1-day Knowledge Translation Training workshops [
20]. Overall, relatively few of these programs provide significant opportunities for interaction or offer a sustained mentoring component [
17].
Several universities offer ongoing support and training through their respective Clinical and Translational Science Award Programs [
14,
21] as well as concentrations in implementation science as a component of their masters, doctoral, or post-doctoral programs [
14,
16]. Online webinars and resources are also provided through institutions such as National Cancer Institute’s Division of Cancer Control and Population Science [
22], the Veterans Affairs’ Quality Enhancement Research Initiative & Health Services Research & Development group through the Center for Implementation Practice and Research Support [
23]. The variety in formats can be helpful to provide access for researchers who otherwise would not have the time or resources to be trained.
Various groups have undertaken work to develop competency and curriculum lists and expectations [
12‐
16]. A few programs have conducted evaluations of their trainings pertaining to knowledge acquisition [
13,
24], although few have assessed skill gains of their trainees over a longer period of time. An evaluation of the IRI using social network analysis found that mentoring was significantly related to outcomes, 2 years later, of collaborations focused on new research, grant submission, and scholarly publications [
25]. The
Implementation Science Editors’ recently issued call to action to build capacity for researchers to be able to conduct D&I research [
17]. In this context, we set out to explore the effectiveness of the MT-DIRC D&I training program.
Purpose of the MT-DIRC training program
Funded by the National Cancer Institute, MT-DIRC is an R25 grant-funded, post-doctoral education program, aimed at building capacity among early to mid-career, cancer control researchers in D&I science through supplemental training and mentorship in D&I research. During each year of the 2-year MT-DIRC program, fellows attended a 5-day summer institute at Washington University in St. Louis to receive didactic, group, and individual instruction on their research area of interest as it pertains to D&I science. Ongoing mentoring relationships occurred over the 2 years of the program, complemented by intermittent webinar sessions with topics chosen by the fellows themselves. This paper describes the various components of the MT-DIRC program and shares preliminary results from the first two trainee cohorts. The purpose of this paper is to describe the development of the MT-DIRC program and share the experience to date to inform the development of other D&I training programs with the aim of advancing the field of D&I research within cancer control and more broadly.
Discussion
Overall, early results from the MT-DIRC program show fellows are improving in their D&I skills and mentoring needs of the fellows are being met. Findings, discussed below, highlight the impact of the program and provide guidance for similar model programs. The emphasis on mentoring and skill development throughout the course of the program we believe has been integral to its success. However, the team has aimed to keep the program flexible to meet the needs of each cohort and the individual fellows and do not believe that all programs must contain every component in order to be successful. Rather, those interested in creating similar programs can utilize these results to inform their decision-making and help shape the content and format for their programs.
D&I skills
All D&I skills showed significant improvement from the time of entering into the program to the 6-month post-institute, with further improvements 18-month post-institute. While fellows showed improvement from their time in the program just within the first 6 months, there was continued improvement in skill levels over the subsequent 12 months suggesting that participants were using the skills they learned at the institute over time which led to continued growth in these areas.
Since the program was aimed at early-career researchers or mid-career researchers looking to switch their research focus into D&I science, it is not surprising that the largest area of improvement for skill levels were previously identified beginner skills (see Table
5 and Additional file
2). And since most of those beginner level skills were located within the “Definitions, Background, and Rationale” domain, we also saw that domain as having the largest overall skill increase. With the field of D&I research growing and its visibility expanding, more fellows are entering the program with a D&I conceptual framework to build upon when they enter the program and are able to master the beginner level concepts more quickly. These results may also suggest that the program does a reasonable job in training beginner level D&I skills, and more focus is needed on addressing advanced-level skills, whether through additional training throughout the year or perhaps even adding a training program in the future that focuses on advanced level D&I skills. However, the level of advancement of the D&I skills over the various time points also speaks well to the results of the original mapping of these competency levels when originally developed [
15].
Mentoring
Our findings show that, in general, faculty mentors rated themselves lower on mentoring skills (MCA) than their fellows rated them, consistent with previous research [
39]. This finding may indicate that the standard mentors hold themselves to may be higher, in comparison to the expectations fellows have for a mentoring relationship in an academic research context. It may also show that the mentor training, while building mentor’s confidence in mentoring skills, also enabled mentors to more critically evaluate themselves [
41]. Fellows, only having received a 1-h training related to mentoring, were perhaps less inclined to evaluate their mentors to the same degree due the narrower scope of training received regarding mentorship competencies. This comparison of mentor self-assessment and fellow assessment allows for identification of future areas to focus on within our program, either by providing feedback for changes to the program structure or greater support for mentors.
While this is a formal mentoring program, mentoring contracts with fellows were not required (though they were encouraged and a template was provided) nor were meeting duration or structures stipulated. The only requirement was that mentors connected with each of their fellows at least once per month, for a recommended time of 1 h. Mentors were given the choice to hold phone calls or skype calls (very few could arrange face to face meetings) and whether they wanted to conduct group calls with all their fellows and/or individual calls. These elements of flexibility may suggest that perceived capabilities of the mentor may vary based on the expectations set out from the initial contact between the mentor and fellow pair. As noted in other research, codifying the mentoring relationship roles, in a way in which calls upon fellows to also be active participants in articulating their needs, may enhance the quality of the relationship and improve perceived success of the relationship [
42].
The questions in the Mentoring Priorities and Satisfaction section of the survey showed mismatches among the specific priority areas of skill building and sharing resources (see Fig.
2 for a visual depiction). This mismatch reflects that fellows were highly satisfied with these priority areas but considered them less important than other priority areas. Another mismatch at 6 months was the high importance of “skill building” (
μ = 4.50) coupled with the lower satisfaction among fellows (
μ = 3.81). This alignment levels out by the 18-month mark, suggesting that some skills and priorities may take more time to develop and conceptualize than others. However, during the 18-month assessment of satisfaction, the mean score of each priority area shifted upward, indicating a general positive shift in all areas of the mentor priority areas. So, even if there was no exact alignment of mentoring areas between priority and satisfaction, overall, the fellows indicated higher levels of satisfaction with their mentoring toward the end of their time in the program.
Other research has found that fellows desire greater emphasis within the mentoring relationship on broader topics such as personal-professional life balance, whereas our findings show that fellows typically rate these psychosocial topics as less important while being highly satisfied with the support in these topics [
43]. In particular, “emotional support” is listed as the least important priority for fellows but is rated highly in terms of satisfaction. It may be that expectations in these areas are lower than that of others. Since the mentoring received here is of a formal nature and geared specifically toward D&I capacity-building in cancer research, it may be that the fellows are already receiving the psychosocial support they need from previously established mentoring relationships at their home institutions.
Limitations
There are some limitations in the way that skill assessments were conducted. First, the assessments of fellows’ skills were self-rated. Fellows were asked about “How skilled do they feel” for a specific competency at that particular point in time. The project team has suggested administering an objective skill test to fellows at the end of their time in the program. However, there is currently no known validated, objective D&I research skill test.
The relationships between the fellows and their home mentors were also not explored in depth. All applicants to the program were asked to identify home mentors in their applications, but there was no formalized process to connect home mentors to the assigned MT-DIRC mentors. This would have been an interesting relationship to follow to see if it produced higher satisfaction with mentoring or increased D&I skills. There is also the issue of power dynamic when assessing mentor’s skills by the fellows. While individual mentoring data are never shared, fellows may be unlikely to rate their mentors too critically given that these mentors serve in this role outside their normal research duties.
Future directions
While additional data will be gathered that may strengthen or modify these results, we see the need to present these early findings as soon as possible to help inform the next set of D&I research training programs in their development. Final data collection for all participating cohorts will be completed with the 18-month assessment in January 2019. At that time, our team will assess the overall impact the program has had on all 56 fellows who have been enrolled. Additional case studies and qualitative analysis will help provide more robust feedback on the overall impact the program has had on the individual fellows and their career trajectory. Following methods used in the Luke et al. [
25] evaluation of the IRI, the team plans to continue to collect data regarding the fellow’s academic output as well as conduct a Social Network Analysis of fellows’ professional relationships. There are plans for future work to connect the outcomes between a fellow’s social connectedness and their skill level.
Additional work can also be done to compare the effectiveness between the different D&I program formats. While this program currently uses its own evaluation tools, these tools can be formalized to be useful across disciplines and formats. This additional comparison could provide support for the range in the types of D&I training formats and assess which ones are suited to particular populations (e.g., students, post-docs, assistant professors, tenured associate, or full professors) or learning styles.
The results from this program are valuable in demonstrating the effect and need for such training programs. The experience with MT-DIRC over the past few years provide a sound understanding of the landscape and a solid infrastructure that can be carried forward to fit with the new training models supported by sponsoring agencies. Results like these demonstrate promising evidence of the effectiveness of such mentored training programs and support the funding of such programs in the future. It is the hope that those who intend to replicate D&I trainings can apply many of the principles outlined to develop and evaluate related training programs.
Acknowledgements
We thank Matthew Kreuter, PhD; Graham Colditz, MD, DrPH; Anne Sales RN, PhD; and Debra Haire-Joshu, PhD, for their service as the core faculty members to the MT-DIRC program. We are also grateful to Melissa Franco, BS, for her assistance in supporting MT DIRC.