Navigating barriers: two-year follow up on recommendations to improve the use of maternal health guidelines in Kosovo

The aim of the current study was to conduct a qualitative process evaluation, using a realist evaluation approach [15], of activities and progress on the nine recommendations made to prepare Kosovo to implement the WHO prevention and treatment of PPH guideline. The period of interest for this evaluation was from October 2012 until November 2014. The specific objectives were to: 1) understand progress to date on each 2012 recommendation; 2) identify barriers and facilitators to progress on each recommendation; 3) capture stakeholders’ proposed future directions; and 4) develop refined recommendations to facilitate implementation.

Focus groups and interviews were conducted in Kosovo with support from the WHO Country Office in Pristina, Kosovo; the United Nations Development Programme (UNDP)/United Nations Population Fund (UNFPA)/United Nations Children’s Fund (UNICEF)/WHO/Special Programme of Research, Development and Research Training in Human Reproduction (HRP), Department of Reproductive Health and Research, World Health Organization Headquarters in Geneva, Switzerland; and the KT Program at St. Michael’s Hospital in Toronto, Canada. Focus groups were conducted during an in-person meeting held in Pristina, Kosovo in 2014.

Local stakeholders (e.g., policymakers, researchers/academics, frontline health care providers, and representatives from professional associations and non-governmental organizations) who took part in the two-day workshop in 2012 were invited to participate in a focus group. They were told that they would be asked to discuss progress, challenges, and opportunities related to the nine key recommendations made to improve the uptake of the WHO guideline recommendations for the prevention and treatment of PPH. Additionally, five local stakeholders were identified as key informants on maternal health in Kosovo and were invited to participate in one-on-one phone interviews. Participants were identified because they play a key role in guideline development and/or guideline implementation; two of the four participants were also present at the in-person meeting.

Facilitators and interviewers used a semi-structured guide during both the focus group (FG) session (which lasted approximately 60 min) and interviews (which lasted approximately 45–60 min; Additional file 1). Participants received a summary of the 2012 workshop findings and recommendations to review before the FG/interview (Additional file 2). The summary document included a column where participants had the option of anonymously commenting on the progress made on each recommendation. The FG session and one-on-one interviews were audio recorded. The FG session was conducted in Albanian and recordings were simultaneously translated into English and transcribed by a translator. Interviews were conducted in both English and Albanian and were transcribed verbatim or simultaneously translated and transcribed by an independent translator, respectively.

Two qualitative analysts (not involved with data collection) independently performed a qualitative analysis of the transcripts by using a thematic analysis approach [17]. First, the analysts familiarized themselves with the data to develop coding themes. Second, these themes were refined into categories, which were then compared, sorted, and grouped to develop a framework of findings. Third, all transcripts were imported into qualitative analysis software (NVivo 10) and the analysts independently coded using the developed framework. Inter-rater reliability (i.e., the degree of agreement between two coders) was compared by calculating percentage agreement within NVivo 10; any discrepancies (i.e., < 80 % agreement) between the analysts were reconciled through discussion. Using the technique of integration, FG and interview data were considered together to draw meaningful and pertinent recommendations that are feasible and relevant for the Kosovo context. Revised recommendations were developed collaboratively with the study authors, including end-users (Kosovo partners), guideline development experts (WHO partners), and KT experts (SMH partners).

Table 2 presents the findings of the process evaluation conducted in 2014 organized by the nine recommendations identified in the original workshop in 2012. For each recommendation, we have presented progress to date, barriers, facilitators, and participants’ proposed future directions. The information provided in Table 2 represents the aggregate perspectives of participants in the FGs and interviews.

Participants reported that substantial progress had been made on three recommendations. First, significant progress has been made in developing the centralized system for data collection across clinical settings (recommendation 1). Although the system has not yet been established, a number of steps have been completed and the initiative has gained leadership buy-in. Specifically, it has been added as a priority to the Ministry’s Sectorial Health Strategy 2014–2020. The Electronic-Based National Health Information System is being piloted in three Regions of Kosovo (Pristina, Prizren and Ferizaj) in all three levels of care. This pilot is a necessary pre-requisite to implementation of the Health Insurance System, planned for 2016. They are currently determining the most important and priority SMART (Smart, Measurable, Attainable, Relevant, Trackable) performance indicators to initiate payment for performance, as next step toward improved quality of care. Pay for performance was meant as motivating factor for improved individual and institutional performance and creating competitive health care environment in terms of higher level and/or better quality of clinical guideline implementation. Second, progress has been made in incorporating standards into clinical practice using a monitoring system for guideline adherence (recommendation 2). These efforts have, however, been made primarily at a local rather than national level; the current goal is to scale up these efforts. Barriers to scaling up these monitoring standards nationally include lack of awareness, resource issues (human and financial), and evaluation challenges. Participants proposed several future directions to address these barriers and facilitators. Third, Kosovo stakeholders have successfully reached out to neighbouring countries to provide mutual support for guideline development and implementation (recommendation 9). Participants identified several future directions for building partnerships with other countries.

Participants indicated that limited progress had been made on four recommendations. Although some workshops had been offered, participants saw a need for additional capacity building and support after the training (recommendation 7). Participants perceived lack of funds for training as the primary barrier to capacity building and believed that there were opportunities for participants to support each other after attending workshops. For the recommendations that involve developing working groups (recommendations 5, 6, and 8), participants thought little progress had been made over the past two years, primarily due to system-level barriers.

Two recommendations describe a cultural shift in perspectives that may enable guideline implementation; however, these recommendations are not as readily actionable as are some of the other recommendations. Specifically, there is a need to create motivational strategies, such as incentives for health care staff, to encourage guideline adherence and enhance communication across stakeholder groups, including clinicians, managers, and policymakers through participation in such activities as guideline development committees (recommendations 3 and 4). Stakeholders made less progress in both of these areas and identified several barriers that may have prevented progress.

In October 2012, we conducted activities to plan the process of implementing the WHO guideline on the prevention and treatment of PPH in Kosovo. Nine key recommendations and a supporting report on findings were developed as a result of the activities [11]. Two years after the workshop, we completed a realist process evaluation using FGs and key informant interviews to assess participants’ perspectives on progress on the nine recommendations to date, barriers and facilitators experienced related to implementing each recommendation, and possible future directions. The realist approach to the process evaluation allowed us to understand how the characteristics of the implementation setting affected the outputs and progress reported by participants in relation to operationalization of the nine recommendations [15]. One of the lessons learned from the first workshop in Kosovo, and from subsequent similar activities conducted in other LMICs [18‐20], was to develop concrete and actionable recommendations for implementation that are grounded in behaviour change theory [21]. Therefore, we developed five updated recommendations: a) develop a guideline development/adaptations working group to create or adapt maternal and perinatal health guidelines; b) develop a guideline implementation working group to pilot test an evidence-informed implementation strategy to support guideline implementation, then scale up the initiative; c) develop guideline policy working groups to work collaboratively with MOH and clinician stakeholders to identify and support opportunities to engage in policy changes related to guideline implementation; d) continue to develop and operationalize a centralized system for data collection; and e) expand guideline development and implementation capacity building and networking activities. Table 3 presents five recommendations adapted from the original recommendations based on progress to date, barriers and facilitators, and suggestions made by participants for improvement. To ground the work in theory, we have also presented elements of the behaviour change wheel [21] that are relevant to each stakeholder group.

These recommendations reflect the planning phase of implementation, preparing the system and organizations for guideline implementation, also referred to as readiness, "the extent to which organizational members are both psychologically and behaviorally prepared to implement change" [22]. Although groups may wish to jump into implementation, process models explicitly describe readiness as a step in implementing evidence in practice [12, 23, 24], given that readiness predicts implementation and outcomes [25‐27]. In addition to understanding organizational and individual readiness, in LMICs there is a need to also focus on system readiness [26]. Infrastructure (or lack of infrastructure) can facilitate or hinder implementation and outcomes; therefore building a strong infrastructure can have long-term benefits [28, 29]. There are tools to address system readiness in LMICs which can help in this process, such as the Intervention and Research Readiness Engagement and Assessment of Community Health Care (I-RREACH) [30].

In reviewing the progress to date on the recommendations, we noted that two of the recommendations involve more of a cultural/system-level shift and are, therefore, more difficult to operationalize. Specifically, the in-country work in 2012 identified two significant barriers to implementation: lack of communication between key stakeholder groups and a focus on punitive interventions (rather than taking a more positive, motivational approach) [11]. Both of these barriers are likely influenced by the socio-political climate in Kosovo since the disintegration of Yugoslavia [31], and therefore are not likely to change quickly, nor are they entirely within the sphere of influence of the key stakeholders involved in guideline implementation. It is important to consider the multi-faceted and interactive effects of personal and environmental factors that determine behaviours [32]. Given this context, we have added two cross-cutting principles that all working groups should consider when undertaking guideline development and implementation work:

The findings of this process evaluation are transferrable to other areas of maternal health work and implementation of evidence-based guidelines, particularly in LMIC settings. Many of the barriers and facilitators experienced by health care system stakeholders in Kosovo also exist in other contexts [33], and the need for specific and actionable recommendations to operationalize implementation efforts is crucial. Recommendations should clearly define the “who”, “what”, “when”, and “how” of an implementation strategy so that stakeholders understand their roles, the objectives, the mechanisms of change, and the anticipated outcomes.

Enabling systems change is challenging [34], and the addition of system-level conflict and instability, which are a reality in many LMIC contexts, exacerbates these issues. Given this context, it is imperative to build strategic partnerships across sectors and levels of government, advocating for policy change, and building the capacity of health care providers and other health care system stakeholders. Existing KT platforms that provide guidance and support on policy engagement can help support these efforts, such as EVIPNet and the Supporting Use of Research Evidence (SURE) [35]. These can be particularly helpful in thinking through how best to communicate with stakeholders (e.g., policymakers) and how to rapidly respond to policy requests [36, 37, 3]. Additionally, ongoing implementation and technical support for working groups in LMICs (i.e., training and coaching) can be provided by groups like the GREAT Network to aid in-country stakeholders in achieving and sustaining implementation goals [38].

Given that research and implementation activities largely stem from time-limited grants, little research is done to evaluate the long-term activities and impacts of implementation efforts. Therefore, this study presents a unique perspective on implementation activities by following-up on activities two years after the initial workshop. However, some limitations of the study should be noted. Although we were able to collect data from a large number of participants, the size of the focus group was larger than ideal [39]. We had the opportunity to include a wide range of stakeholders in the focus group by combining the focus group with an existing meeting; however, this resulted in a large number of participants, all of whom may not have been able to express their views. Consequently, we decided to conduct interviews with a few key stakeholders to make sure that we captured their perspectives. In addition, communication challenges between groups were identified as a major barrier in 2012 and continue to pose challenges. We had limited MOH representation and involvement. Ideally, we wanted to collect data from the MOH perspective as they are a driving force or barrier behind many of the activities but they declined participation. Finally, our understanding and interpretation of the data were limited by cultural barriers and local contextual factors. To reduce the impact of this limitation, an in-country expert was consulted throughout the process to enhance comprehension of the data and its relevance to the local context; this individual is included as a co-author on this paper.