Supporting collaborative use of the diabetes population risk tool (DPoRT) in health-related practice: a multiple case study research protocol

DPoRT, developed by Rosella and colleagues [10], is a novel tool that offers an innovative way to make routinely collected population health data practical and meaningful for diabetes prevention planning and local decision making. The tool was designed to apply routinely collected and publicly available data, enabling its use by different users (e.g., epidemiologists, public health decision makers). To accomplish broad applicability, the tool works on risk factor data available to users at community, regional and provincial levels (i.e., Canadian Community Health Survey [CCHS]). A recent systematic review of all diabetes risk tools found that of over 90 existing tools, DPoRT was the only one built specifically to inform population intervention strategies [6]. DPoRT was also the only tool rigorously validated in two diverse external populations and across important subpopulations (e.g., ethnic groups) [11]. Specific uses of DPoRT include identifying at-risk groups, generating intervention scenarios that allow the user to weigh trade-offs of strategies, and estimating future number of diabetes cases and costs for the purpose of health resource planning. Despite these significant advantages, the uptake and utilization of DPoRT in health-related settings remains sparse.

Research has shown that passive knowledge translation (KT) strategies are likely insufficient to maximize uptake and utilization of knowledge products and innovations in various sectors [12‐16], including public health [16]. In our experience, conventional researcher driven KT approaches [17] are insufficient for facilitating the use of tools such as DPoRT and the tailored knowledge products that result from such use. Furthermore, simply providing health-related organizations with DPoRT and letting them figure out how to use it will not effectively influence practice and decisions because of perceptions that the software must be run by the original developers. Since DPoRT and its proposed applications are novel, specific KT strategies to inform how to optimize its uptake and utilization are lacking. Nevertheless, interactive and integrated KT approaches have shown promise as a means to capitalize on the reach and potential of innovations [14, 18‐20], particularly when knowledge users are included as integral partners in the process [21‐23]. Consequently, this study intends to facilitate the collaborative use and application of DPoRT by key decision makers and researchers through an integrated knowledge translation (IKT) approach.

We are not aware of any published studies of KT approaches specifically designed to support the application of tools designed to generate future population-level risk profiles to facilitate decision making. Work in related domains, including clinical decision-rules [24, 25] and guidelines [26], demonstrates the importance of KT strategies that are tailored to both the specific properties of the tool as well as the intended users. In an effort to address this gap in the literature, we designed an IKT approach that involves partnering with regional and provincial health decision makers in two provinces to facilitate the uptake of DPoRT using several different strategies. The execution of this IKT approach is informed by the Knowledge-to-Action (KtoA) framework developed by Graham [15].

The KtoA framework describes the KT process as encompassing two key components, knowledge creation (represented by a funnel that distills and tailors information into useable knowledge products or tools) and action (facilitated through a dynamic context-driven process) [15]. The KtoA framework offers a suitable structure for conceptualizing past, present and future efforts to attain effective and contextually relevant applications of DPoRT. The knowledge creation phase was completed through the development, validation and refinement of DPoRT [10, 11, 27]. Furthermore, through pilot work with Peel Public Health in Ontario, we have confirmed that the current form of DPoRT is relevant for application. The action phase of the KtoA framework describes the actual tasks and activities required for implementation and uptake. However, research to inform the KT and application of population-level risk prediction tools (e.g., DPoRT) by knowledge users in health-related settings (i.e., the KtoA action cycle) has yet to be undertaken. Thus, this protocol addresses the process of moving DPoRT into action through the collaborative efforts of researchers and DPoRT users. Our conceptualization of the KtoA framework, adapted for this project, is presented in Figure 1. The details of the conceptual map guiding this work are forthcoming in a separate manuscript.

The overarching goal of this project is for researchers and decision makers in varied health-related settings to work collaboratively to build capacity and facilitate the use of DPoRT as a strategic aid for population-based risk assessment, intervention and planning decisions. In an effort to empower health organizations to use DPoRT for their own decision making and surveillance activities, we have developed an IKT strategy in partnership with regional and provincial decision makers in Ontario and Manitoba, Canada. To achieve this goal, the specific objectives of this project are to:

Health organizations in this study and the populations they serve vary considerably, and there is increasing recognition that differences in context make it difficult or impossible to replicate interventions exactly the same way across different conditions [28]. It has been suggested that we should continue to expect standardization in the process, sequence and function of an intervention, but that we allow for adaptation to the dynamic and unique properties of each context [15, 22, 29‐33]. With this advice in mind, we have outlined a set of activities that we believe must be carried out in each setting to ensure valid, reliable and useful application of DPoRT. However, as we enter into this process and at the outset of each step, key stakeholders will be invited to engage in exchanges and assessments that will help tailor the form of these activities to suit their unique needs and realities.

To facilitate our intended IKT approach, we will use a KB team. In the literature, knowledge brokers have been described as trusted, knowledgeable, skilled and solution-oriented individuals or groups who act as ‘go-betweens’ to bring key stakeholders together in formal and informal venues for the purpose of acquiring, sharing and applying knowledge to improve decision making and/or the management/delivery of services [34‐39]. Knowledge brokering is becoming a popular strategy for linking the producers and users of knowledge to facilitate the reach and effective uptake of innovative research. There is an expanding literature describing the effectiveness of initiatives that have utilized knowledge brokering approaches (e.g., public health [40], mental health [41], community-based non-profit organizations [42], and clinical practice [43, 44]). As supported by other experiences with knowledge brokering in public health [33], we believe this is an appropriate approach to adopt as our primary KT strategy to support all project objectives. However, in contrast to strategies that use a single knowledge broker, we have established a KB team that spans the content expertise areas required to support the uptake and use of DPoRT. Specifically, the KB team consists of: a member of the DPoRT development research team, an epidemiologist trained in using DPoRT, and two research coordinators with expertise in KT and evaluation in health-related settings. The KB team will: provide support in building the partnerships (objective 1), lead, coordinate and monitor training and uptake activities, help to navigate and appraise DPoRT applications, identify barriers, and encourage collaborative, creative and context-specific problem-solving (objective 2). In addition, members of the KB team will facilitate and collect data for the evaluation (objective 3). DPoRT computations are relatively straightforward, but application of the tool may be challenging for the users, and one-off strategies such as a training workshop on how to run the software will not be sufficient to realize the potential value of the tool for informing decision making. The strategy must also consider that some of the users will not be applying DPoRT themselves but rather using the outputs of DPoRT. We believe that sustained partnerships between DPoRT users and the KB team will facilitate DPoRT application and ensure effective uptake and use.

We have designed a utilization-focused evaluation [45, 46] leveraging the collaborative relationship with the DPoRT users. Our approach reinforces the IKT nature of this project such that the KB team facilitating the IKT approach will also be actively engaged in evaluation activities [47], and the DPoRT users will be involved in focusing the evaluation [46]. The evaluation will employ a multiple case study approach [48] with the participating organizations as the units of analyses. The evaluation will occur throughout the action cycle and will be based on analysis of both quantitative and qualitative data collected from passive (observer notes) and active (surveys/interview) methods. This approach is particularly useful for this project because it can be applied when the activities involve complex social interactions, when control over variables is limited, and when the boundaries between the phenomenon under study (DPoRT use) and the application contexts (varied health-related organizations) are blurred [48, 49]. Case study design also allows for an in-depth understanding of the action cycle in each setting, to explore organizational capacity, and to identify the barriers and facilitators to DPoRT use. Within-method triangulation [50] will be used to gather in-depth information at multiple points in time and from multiple sources.

The Research Ethics Board at Public Health Ontario (PHO) has approved this study protocol.

The two participating public health units in Ontario have different internal capacities and serve different types of communities. Peel Public Health (PPH) is a local public health department serving over 1.3 million residents in three primarily urban municipalities. PPH is part of the Health Services Department within the regional government whereby Peel Regional Council serves as its Board of Health. PPH’s 700-member staff provide services specified in the Ontario Public Health Standards and the Health Protection and Promotion Act [51]. Peel is a rapidly growing and highly diverse community with a large immigrant population (49%). Within PPH there will be one main group of DPoRT users: the epidemiology team, which provides synthesized health status data to public health teams to support informed decisions about programming. We conducted pilot work in Peel prior to developing this protocol, which informed our IKT approach as well as training materials.

Simcoe Muskoka District Health Unit (SMDHU) is a local public health unit situated at the north end of the Greater Golden Horseshoe and in the heart of Ontario’s ‘Cottage County’. The health unit’s jurisdiction covers over 8,000 km² and includes 2 upper tier governments, 2 cities and 24 smaller municipalities. In 2006, the total population for Simcoe and Muskoka was 479,767; projections suggest this number will be 40% higher and will include many seniors by 2025. The SMDHU Board of Health is an autonomous governing structure made up of appointees from the County of Simcoe, District of Muskoka, City of Barrie, City of Orillia and Provincial designates. The health unit has 400 staff members. Within SMDHU there will be two main beneficiaries of DPoRT: the Planning and Evaluation team within Corporate Services and the Chronic Disease Prevention Healthy Lifestyle (CDP-HL) team within Healthy Living Services. The Medical Officer of Health, one epidemiologist, and one manager from the CDP-HL team will be the primary participants.

DPoRT was validated in Manitoba because of the different population structure from Ontario, thus demonstrating the robustness of the tool [10]. More importantly, diabetes is a significant health problem in Manitoba, and diabetes prevention and planning is a priority for decision makers. We will engage with several decision makers in Manitoba representing a range of populations, as well as a diverse set of DPoRT users responsible for diabetes prevention and planning. Manitoba Health is a department within the Government of Manitoba which operates under the provisions of the legislation and the responsibilities of the Minister of Health. The legislation, as well as emerging health and healthcare issues, guides the planning and delivery of healthcare services for Manitobans. Manitoba Health is currently undertaking a Chronic Disease Prevention Initiative (CDPI), which will benefit from DPoRT application to inform the most relevant strategies for diabetes prevention. Regional Health Authorities (RHAs) via ‘The Need To Know’ (NTK) Team: We will apply DPoRT to the local regions in Manitoba via the NTK Team, which is a well-established conduit for KtoA in the province [18, 52, 53]. The three objectives of the NTK Team are to create new knowledge of high relevance to policy and planning imperatives in the regions, to engage in capacity building activities among the partners, and to ensure that the research is disseminated and applied at the regional level. The NTK Team involves scientists from the Manitoba Centre for Health Policy (MCHP), high-level planners from each of Manitoba’s five RHAs and Manitoba Health. The team meets three times a year, for two-day meetings. Team members take on the critical role of knowledge uptake when they facilitate roundtable discussions in the annual MCHP/RHA Workshop Days, where RHA board members, chief executive officers, vice presidents, program managers, Medical Officers of Health, and others examine current MCHP research report findings. These workshops are designed specifically so that RHAs can learn about better ways to use population-based data to guide priorities for regional strategic plans and operations. Disseminating DPoRT to the RHAs via the NTK team is a resourceful way of leveraging an existing and effective KT conduit. Through the NTK team, interested RHAs will be connected to the KB team and invited to participate in the KtoA process. In order to build capacity for DPoRT use, Manitoba Health will work in collaboration with the interested RHAs.

Throughout the KtoA cycle, the KB team will keep a record of the nature and sequence of events occurring at each site. At an initial site meeting and throughout the KtoA cycle, the KB team will inform participating staff about the collection of observational notes, the type of data that will be logged, the reason for recording such notes, and will explain that individuals can opt out if they prefer to not have their interactions with the KB team logged. Potential events that may be recorded include: requests by DPoRT users for technical support or resources related to DPoRT, discussions with the KB team to troubleshoot barriers to DPoRT use, discussions to support the development of knowledge products, etc. The log will record the type of interaction that the KB team engages in (e.g., phone, email, in-person meeting), role of the individual(s) spoken with, type of activity being conducted (e.g., facilitating, preparing, checking-in), length of time spent on interaction/activity, purpose of the interaction, actions taken to address issues/concerns, and outcomes/decisions that followed. The KB team will also keep a reflexive journal of critical reflections on partner interactions and setting processes. The purpose of recording observational notes is to understand the effectiveness and intensity of the knowledge brokering strategy as well as provide feedback on how the KB team can improve, given that we are utilizing an IKT approach.

The KB team will build capacity in using DPoRT at each site by delivering a training workshop. Each organization will determine who will attend training based on their anticipated use of DPoRT; these details will be discussed during initial site meetings. Additional training activities, such as one-on-one support, may be provided. To evaluate the effectiveness of the training activities, the KB team will administer a survey and After Action Review (AAR) [54] to individuals who participated in the DPoRT training workshop. The brief post-training survey will take approximately five minutes to complete, and will be used to assess the effectiveness of training delivery and level of confidence and comfort with using DPoRT independently. The AAR will be completed within two to four weeks of the training and will consist of a facilitated half-hour discussion to gain more details about experiences with the DPoRT training, and to identify aspects that went well or need improvement.

In order to assess the effectiveness of the partnership between the KB team and each site, participants will be invited to complete the Partnership Self-Assessment Tool (PSAT) developed by the Center for the Advancement of Collaborative Strategies in Health [55] based on the Partnership Synergy Framework [56]. PSAT measures team synergy relative to: leadership, efficiency, administration, management, and sufficiency of resources. PSAT is a commonly used and validated measure for assessing the effectiveness of a partnership, which is a critical dimension of our IKT approach. In line with recommendations from the PSAT developers, this measure will be administered approximately six months after the commencement of the partnerships with each DPoRT user site.

Approximately two to three months after DPoRT users have begun using the tool for their site-specific purposes (e.g., surveillance, program planning, etc.), a member of the KB team will conduct semi-structured interviews at each site with those who have used DPoRT (either directly or indirectly via outputs). The purpose of these interviews is to understand the effectiveness and impact of the KtoA process on DPoRT uptake, the effectiveness of the partnerships, and to assess actions, such as training and support, taken by the KB team. The questions in the interview guide will follow the OMRU guidelines [55] and probe elements of the KtoA process cycle [19].

Qualitative analysis will be conducted to identify emerging themes that require further exploration and to inform/or redirect the ongoing KtoA process. To inform the organization and coding of the qualitative data, we will a priori identify sensitizing concepts [48, 50] structured as coding guides from the analytic frameworks guiding this project [14, 15, 56, 57]. In-depth descriptions will be created for each setting that are holistic and context sensitive [48] using descriptive and iterative methods [58]. Additional coding domains will be developed as needed. All coding will be carried out by one individual; a sub-sample of the data will be coded by two independent researchers in order to assess confirmability through analyst triangulation [54]. We will use NVivo9 as a qualitative data management and analysis tool. Quantitative data derived from user surveys and user metrics will be analyzed and managed using SAS statistical software and will include descriptive summaries (such as means, frequencies) and, where applicable, correlations and bivariate statistics.

Since DPoRT users will consist of employees at health-related organizations, some participants may not feel comfortable revealing sensitive information related to their experiences with DPoRT, the training workshop, and the ongoing KB team support. In an effort to enhance participant confidence, letters of information will be provided to explain the measures that the KB team will take to protect confidentiality. For example, identifiable information will be stripped from transcripts, and interviews will be conducted in private rooms. Surveys and observational notes will not record identifiable information.

Additionally, given that implementation process-related data will be collected several months following the establishment of partnerships and training workshops, recall bias may limit the accuracy of DPoRT users’ descriptions of the implementation process. However, the use of multiple sites, and data sources (i.e., triangulation) should mitigate the potential for recall bias [50].