Background
The literature has demonstrated many complex and interconnected factors that can determine implementation success. These include supportive leadership, an enabling organizational culture, patients’ perceptions on the intervention, amongst others [
1‐
3]. It is also acknowledged that these factors interact with one another, although little is known about exactly how context might shift the balance of interconnectivity. The Consolidated Framework for Implementation Research (CFIR) specifically identifies the implementation setting as an important factor, alongside the interplay between individual stakeholders and facilitators [
4]. However, more guidance on this interplay is needed. Implementation science is focused on improving the successful uptake of evidence-based methodologies to improve health care quality and overall effectiveness [
5]. Evaluating unsuccessful implementations can provide lessons that can be applied prospectively to improve the likelihood of intervention success [
6]. However, details of unsuccessful implementations are often under-reported, leaving notable gaps in the literature [
7], negatively impacting the quality of care that patients receive, as well as contributing to increased costs and provider workload [
8]. Motivated by this gap, we conducted a retrospective evaluation on an unsuccessfully implemented chronic disease management program.
The Chronic Disease Management Initiative (CDMI) program was guided by a chronic disease management model proposed by Bodenheimer and colleagues [
9] with six elements: linkage with community resources, buy-in by health care organizations, structured practice teams for chronic care management, self-management support, decision support and clinical information systems that ensured reminders and feedback about patient progress.
CDMI was designed to support patients living with chronic obstructive pulmonary disease (COPD) through an interactive, mobile-based platform delivered in a primary care setting. Participants were given a smartphone for receiving targeted messages from their healthcare providers. As part of the program, patients and providers received a brief training on the use of the device during regularly scheduled visits. Text messages were initiated by a program navigator at least twice a week to reinforce health teaching and monitoring activities. Previous research has shown that some patients prefer text message communication to in-person or phone conversations because it can diminish feelings of embarrassment related to health issues [
10]. Another study identified that being able to contact health care providers with a mobile device was like having a “permanently outstretched hand”, even when health care services are not being accessed [
11].
After a 10-month implementation period, the CDMI program was unable to enroll a sufficient number of patients to realize any improvement in patient outcomes. As such, researchers were unable to determine whether the smart phone technology, if implemented properly, could actually work to improve communication between patient and provider and, in turn, improve patient outcomes.
The contributions of this study to the implementation science literature include: 1) providing experiential knowledge to guide future health interventions in planning for real world challenges they may encounter, especially on the individual or organizational level; 2) highlighting the importance of involving participants from the outset of planning as it can improve the likelihood of a successful implementation and guide the intervention to fruition; and 3) bridging the gap in the literature where unsuccessful implementations are underreported.
Methods
Our research was guided by the Consolidated Framework for Implementation Research (CFIR) which helped to identify critical constructs in the implementation process [
12]. CDMI’s implementation team (
n = 11) was made up of researchers (
n = 2), family physicians (
n = 2), a respirologist (
n = 1), nurses (
n = 2), psychiatrists (
n = 2), a cardiologist (
n = 1), and a research coordinator (
n = 1).
The 10-month intervention period was led by the research coordinator and executed by a subset of the research team. After the unsuccessful implementation, all eleven team members were invited to take part in a semi-structured interview. Four informants participated in-person (36.36% response rate), one participated via email; one explicitly chose not to partake since they believed that they were not integral to CDMI’s implementation; the others absent did not cite a reason for their lack of participation.
Participants were asked eight open-ended questions about their view on the project’s development and implementation, their involvement, team perceptions, as well as areas of improvement (See Additional file
1). Questions were developed to explore the constructs in CFIR [
12]. The interviews were transcribed verbatim; NVivo10 was used to support analysis.
The interview data was coded by two researchers (SLS, PC) using a conventional content analysis [
13]. The codes and themes were reviewed and refined by the entire research team. Analysis began with an initial read-through of the transcripts to identify significant and relevant content. This process was repeated until the transcripts were fully coded and all relevant content was marked. Data analysis was augmented by looking at related implementation documents such as meeting minutes, project proposals, and ethics documents. Codes were combined into themes and meaningful patterns in the data were examined in relation to all data sources [
14]. The STARi checklist by Pinnock and colleagues [
15] was used to ensure transparent research reporting (See Additional file
2).
Discussion
The themes gathered from this study may seem intuitive and they have certainly been demonstrated as important in the literature, however they highlight the complexity and interconnectivity of these factors in practical application. They also demonstrate the need for a strong implementation plan to guide both program planning and implementation processes. Even when individual implementation barriers are accounted for, complex and interconnected barriers may still arise. This is especially true if the proper and continuous engagement of all key stakeholders is not done well. For our study, it was clear that having an interdisciplinary research team was necessary for enhancing the planning process; however, it was not sufficient for identifying all barriers and it may have hindered implementation success.
A poor theoretical basis for implementation guidance can make retrospective analysis of failed implementations difficult [
16]. When implementation is guided by theory, it is more likely to succeed. The CDMI program was theoretically grounded during its planning stages [
9], however, the implementation phase lacked theoretical guidance. Even though five of the six theoretical components proposed by Bodenheimer and colleagues [
9] were considered in planning (buy-in by health care organizations, self-management support, structured practice teams for chronic care management, decision support and clinical information systems [
9]), they lacked practical and contextual consideration for implementation.
With the exception of linkage with community resources, all other elements suggested by Bodenhiemer were incorporated into program planning. First, buy-in was achieved during program planning through an interdisciplinary team, with active participation from family physicians, and specialists in COPD, as well as psychiatry. Second, a central feature of CDMI was a navigator role to enhance patient self-management support and to support program implementation. This self-management support, although well intention, did not work well in practice largely due to insufficient consultation with providers and lack of patient engagement in planning. Third, structured practice teams were enabled through the proposed coordination of the navigator and primary care provider, with guidance from specialists, to share the responsibility of assessing and monitoring patients’ through the smartphone platform (including symptoms of depression, anxiety, adherence to medications and follow-up appointments). Fourth, decision support was incorporated through the navigator role who was meant to be available to support implementation and any technology trouble-shooting. Lastly, clinical information systems were a key design feature of the program; existing electronic medical records system were integrated with the programs text messaging to provide reminders to patients and feedback to providers. Linkage with community resources was not considered during program planning or implementation. Our study identified the several shortcomings in the application of the proposed constructs by Bodenheimer and colleagues [
9]. First, the lack of patient engagement and provider consultation meant that the program was not aligned with the needs patients or requirements of providers. Second, while buy-in from organizational leads was obtained in planning, similar buy-in from front-line providers for implementation was not achieved. Third, decision support and clinical information systems were felt to duplicate existing services without clarity on expected outcomes for patients or providers. Lastly, interdisciplinary collaboration was, for the most part, only done at the planning level, and less so at the implementation and patient care level.
There are several implementation models that could have been used to support the success of CDMI such as CFIR or the PARiHS (Promoting Action on Research Implementation in Health Services) framework. PARiHS defines successful implementation as a function of three factors: evidence, context, and facilitation [
17]. Adherence to this framework may have facilitated the necessary ‘pre-work’, such as conducting a proper needs assessment with patients, staff, and other stakeholders. Ecological theories of implementation such as the Active Implementation Framework [
18] or Durlak and DuPre’s Ecological Framework [
19] support a strong consideration for adaptability to factors like multiple stakeholders, complexities of health care systems, and the interconnectedness of variables [
20]. Using a model, theory, or framework to practically support implementation with consideration for context is essential.
The benefits of interdisciplinary collaboration in research and program implementation are noted in the literature [
21]. The CDMI research team was interdisciplinary, however, a lack of previous working relationships led to challenges in implementation. While interdisciplinary teams can support the likelihood of creating a successful intervention [
22], equally as important is ensuring strong working relationships throughout implementation, building on previous successes (when available), and minimizing the risk to momentum if or when there is team member turnover.
There are several limitations of this study including the small sample size and the specific context in which the research took place. Having more study participants could have helped us develop a more complete picture of why this evidence-based program failed to successfully implement. Our findings are not meant to be generalized to other contexts, instead we believe our rich description of this unsuccessful implementation can provide lessons for other interventions – most notably, the importance of using theory to guide and support both planning and implementation, along with the importance of involving all stakeholders in both of these processes.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.