Evaluation of a knowledge transfer scheme to improve policy making and practices in health promotion and disease prevention setting in French regions: a realist study protocol

In France, there is no formal and structural KT scheme. There are a few initiatives led by the National Public Health Agency (ANSP) and the National Cancer Institute (INCA), which for instance produce literature reviews. But policy-makers and prevention professionals do not use them. It confirms that a passive diffusion of knowledge is not effective, and the effectiveness of KT strategies depends on the context in which they are implemented [6‐10]. The contextualization of the KT strategies is necessary to remove barriers to knowledge use. According to Gervais et al. [3, 11], KT research on decision-making processes offers a number of explanatory factors which may be classified in three categories. The first relates to the specific properties of the evidence itself: nature, availability, accessibility, quality and credibility (data and sources), intelligibility, ability to meet needs, adaptability and transferability [3]. The second category relates to the personal characteristics of decision-makers: beliefs or personal values, political leanings, socio-demographics, level of education, previous experiences, motivation and ability to interpret data, etc. All of them may influence how new knowledge is addressed in the decision-making process.[12]. The third category refers to the characteristics of the organisations and local contexts in which knowledge producers and users work [4]: openness to change, material, human and financial resources available for KT, social and political context in the external environment, style of management, leadership, staffing, stakeholder coalitions, etc. Consequently, the multiple barriers to the adoption of evidence in the field of public health underline the non-linear process between knowledge production and knowledge use. If these barriers are to be overcome, we need to address all the parameters that affect the decision-making process. This is a focal point for KT research.

Various strategies are available to overcome barriers to the use of KT. A recent work conducted by Langer et al. identified six mechanisms involved in effective KT:

The authors underline that these strategies are effective if combined and contextualized in their implementation setting, confirming previous work of Ridde et al. [13] and Barwick [14]. Consequently, we hypothesize that in France, as elsewhere, simple diffusion and “one size fits all” strategies are not effective.

In this paper, we present the protocol of a realist evaluation study of knowledge transfer strategies implemented in the field of health prevention at a local level in France. We have used RAMESE II reporting standards for realist evaluations [15].

This study is a comparative multiple case study of a KT scheme in the field of health prevention using a realist approach [16, 17]. It concerns French public health services: ARS and IREPS. The case study design is the more suitable research strategy to investigate a phenomenon within its context and analyse this phenomenon’s interactions with several other elements relevant for our area of study [18].

The realist approach [17] is increasingly used for appraising the interactions between an intervention, its mechanisms and its contexts. The overall aim is to achieve a better understanding of an intervention’s success factors and how these may be replicated in other contexts. This type of evaluation examines what works, under what conditions and for whom, based on a middle-range theory (or configurational theory) which describes the interactions between outcomes, mechanisms and contexts [17, 19]. Thus, realist evaluation integrates the paradigm of black box evaluation [20]. While the experimental paradigm evaluates effectiveness without appraising an intervention’s mechanisms of impact, realist evaluation answers the following question: did the intervention work according to the theory underpinning it? This type of evaluation seeks to understand the intervention by focusing on its mechanisms and the influence of context. The mechanism is defined in this case as the “part of a participant’s response to an intervention, generally hidden and sensitive to variations in context, and which produces effects” [21]. In realist evaluation, causality is generative, meaning that what generates the effect relates specifically to the interactions between context and cause (here, the intervention methods) [19]. However, as we will study the patterns between these interactions in different contexts, we hypothesise that it is possible to isolate key elements that may apply across a set of contexts. These findings will thus generate intermediate theories that will be sharpened little by little as each case will be investigated.

To conduct a realist evaluation, we alternate theoretical and empirical stages (Cf. Fig. 1: The realist sequences). According to Langer’s work [2] and many authors [7, 10, 13, 14, 22], we hypothesize that an effective KT scheme has to combine an access to and an adaption of knowledge, the development of professionals’ skills to analyse, adopt and transfer knowledge into their contexts, the improvement of organisations and processes in order to facilitate the integration of knowledge. We also conducted an exploratory qualitative study in the four regions to collect data on the pre-existing scheme and activities related to KT and the potential local barriers. The questions were the following: what kind of KT activities are possible (types, timeline, duration, management)? Who may be involved? What structural/organisational mechanisms would be affected? What contextual factors, outside the control of those involved, would need to be addressed? Data will be collected by means of semi-structured interviews with IREPS directors and ARS public health directors (8 people). Based on the behaviour change wheel theory [23] and an exploration of the behavioural theories used in KT strategies [24], we hypothesize that the change of which will occur in knowledge use may be notably due to the motivation to use knowledge, the perception of its usefulness and practicality and the ability to adapt it in to fit different settings.

According to all the above scientific literature, and to support our realist evaluation, we built an initial middle range theory, defined as following: “An effective KT scheme combines a relevant access to and an adaptation of knowledge, the development of professionals’ skills to analyse, adopt and transfer knowledge into their contexts, the improvement of organisations and processes in order to facilitate the integration of knowledge. These strategies influence the motivation to use knowledge, the perception of its usefulness and practicality, the ability to adapt it to fit different settings, and produce evidence informed decision-making and practices”.

This initial middle range theory leads to the design of four theories of intervention one for each region describing the interventions, the expected outcomes, the contexts’ parameters and the expected mechanisms. This work will be conducted in a preliminary 2-day workshop, gathering ARS and IREPS professionals. These theories will be applied in the 4 regions, for 12 months, and data will be collected in order to characterize the contexts, mechanisms and outcomes and to determine the effective CMO configurations.

Finally, a cross-sectional analysis of the case studies will be conducted allowing us to identify potential regular CMO patterns, which would constitute an adjusted middle-range theory. The different stages are presented in Fig. 1 (Cf Fig. 1: The realist sequences).

The intervention is a KT scheme designed for the use of policy briefs (PBs), which will be written on the basis of six reviews of systematic reviews (completed by international guidelines); an international scientific committee have carried out these reviews. They concern the following themes: nutrition, alcohol, tobacco smoking, physical activity, emotional and sexual life and psychosocial skills. These themes are primary in France. They present effective prevention practices.

Based on the report entitled “The science of using science: researching the use of research evidence in decision-making” [25], the scheme combines the following activities:

According to the initial middle range theory and the 4 theories of change, professionals will make an action plan to apply them in their local settings. Theses KT activities will be set up over a 12-month period.

The targeted population is composed of prevention and public health services operating in French regions, namely, the ARS and IREPS and their partners. The study will focus on three groups of stakeholders:

We already have the agreement for the data collection given by the four ARS involved in the project since it began.

Data will be collected to document the support scheme’s mechanisms and contexts parameters involved in effectiveness. They will be collected before the implementation of the KT scheme at the end and throughout the implementation. They will be collected on the 3 categories of people described before; 20 people per each region (80 at all).

Collected data will characterise the context, the mechanisms relating to the organisation and to the individuals involved, the PBs and the set-up for KT.

A description of data collected and how and they will be collected are presented in Tables 1 and 2, but these variables will be adjusted according to the four theories of intervention and the action plans (Cf. Table 1 : Expected outcomes and Table 2 : Contexts and mechanisms expected).

Data will be collected by means of:

The observation and documentary analysis grids and the interview guideline will be designed based on the four intervention theories and tested on a sample of five stakeholders not involved in the process, but belonging to the IREPS network.

The collection will last 12 months.

Data will be processed through a content analysis [26] defined as “A set of systematic and objective procedures for analysing communication processes in order to obtain indicators (quantitative or not) inferring knowledge related to the conditions (inferred variables) under which meaningful information is both sent and received”. This analysis will code, classify and grade content in order to identify patterns, trends and specific features. We will use a software program called Nvivo to assist us in conducting and integrating a thematic analysis of the interviews and an analysis of the observation reports. The qualitative analysis will lead to:

Based on the observed elements, we will classify the outcomes in three categories of use as recorded in the literature [27, 28].

We will process these data to characterise the mechanisms identified in the intervention theory and induced by the intervention. This will enable us to answer the following questions about several mechanisms related to knowledge, people and organisations: (1) were they present in the contexts studied? (2) Did they positively or negatively influence the outcomes from a user’s perspective? (3) Which mechanism(s) was (were) actually active (which parameter influenced which other parameter and/or which outcome)? (4)Which outcome(s) was (were) produced?

Then, a secondary middle range theory will then be developed, leading to future guidelines.

Based on this analysis, we will compare the CMO configurations, which will be identify with the initial middle range theory, and we will figure out an adjusted middle range theory. This work will be conducted during a second interdisciplinary workshop, based on a discussion about analysis from data collected, gathering all ARS staff responsible for prevention and public health, IREPS directors and project officers, plus agents from ANSP and INCA.

KT development guidelines in France will emerge from this meeting.

Different types of actors will be involved throughout the study: prevention professionals, policy-makers and researchers. Thus, multiple methods will be used to communicate research results: