Longitudinal assessment of the association between implementation strategy use and the uptake of hepatitis C treatment: Year 2

Hepatitis C virus (HCV) is a leading cause of liver cancer and liver failure in the USA [1]. In fiscal year 2015 (FY15), new, highly-efficacious treatments for HCV became widely available as the evidence-based practice for curing HCV [2]. Prior treatments included injected interferon, which was suboptimal because of side effects, contraindications, and poor efficacy despite year-long treatments. The newer medications included pill-only regimens with minimal side effects, short courses, and high cure rates. As the largest provider for HCV nationally, the Department of Veterans Affairs (VA) sought to spread this innovation rapidly across the country by developing the Hepatitis C Innovation Team (HIT) Collaborative. Funded by VA leadership as a 4-year, national initiative, the HIT Collaborative supported the development of regional teams of providers with the goal of promoting the uptake of evidence-based HCV care throughout the VA. The HIT Collaborative included the components of learning or quality improvement collaboratives [3], such as using in-person learning sessions, plan-do-study-act cycles, team calls, email/web-support, external support of active data collection, feedback and education by experts and Collaborative leadership, and outreach to local and national leadership.

Together, the availability of new HCV treatments and VA’s implementation efforts to increase their uptake resulted in a dramatic increase in treatment and cure of HCV in VA. While only 10% of Veterans with HCV infection had ever been cured of HCV as of the end of FY14, by the end of FY16, 43% or 84,192 Veterans were cured, representing a fourfold increase [4].

While this rapid, national implementation effort has been a tremendous success for VA, it has also provided the opportunity to study the use of implementation strategies and their association with a measurable clinical outcome over time and on a national scale. We previously reported on the associations between implementation strategies and HCV treatment starts at the site level and the extent to which strategies were related to HIT activities in the first year of the HIT Collaborative [5]. To frame our evaluation, we used expert-based definitions of implementation strategies, or methods to increase the uptake of evidence-based practices [6, 7], from the Expert Recommendations for Implementing Change (ERIC) project. ERIC defined 73 individual strategies [8] and then used a mixed-methods process called concept mapping [9] to develop conceptually distinct clusters of the strategies [10]. As the Collaborative continued, we had the opportunity to study how implementation strategy use changed over time within a nationwide healthcare system, particularly in the context of a learning collaborative [3].

This evaluation aimed to document (1) how reported implementation strategy use evolved over the first two years of the HIT Collaborative, (2) the changes in the associations between implementation strategies and clinical outcomes over time, and (3) the role of the HIT Collaborative in implementation strategy uptake.

We previously examined self-reported use of implementation strategies in a national sample and found that the total number of strategies used by a site was associated with the clinical outcome of HCV treatment starts [5]. In this study, we further investigated the use of strategies over time and the associations between site-level implementation strategy use and treatment starts over time. While many of the strategies did not change in use from Year 1 to Year 2, there was a significant increase in the following specific strategies: tailoring strategies, promoting adaptability, sharing knowledge, and using mass media. Moreover, while the total number of strategies used was associated with increased HCV treatment in each year, the specific strategies associated with treatment starts varied over time.

The EPIS Implementation Framework posits that implementation happens along four phases: Exploration, Preparation, Implementation, and Sustainment. The implementation strategies that are appropriate may vary by the stage of implementation [15]. These data support that the implementation strategies associated with successful implementation of a clinical innovation change over time. When the oral HCV medications/clinical innovation first became available, successful sites focused on preparation or “pre-implementation.” The associated implementation strategies included training and education, as well as developing stakeholder interrelationships and seeking interactive assistance. After sites had established the necessary education and relationships, the most successful sites then focused on iterating these and adapting to the context in Year 2. In other words, the strategies associated with treatment shifted across the ERIC group’s concept map between years. The geography of this concept map was developed by implementation experts considering the global similarities of the strategies. The present data suggest that clusters and strategies within clusters may be differentially relevant based on the phase(s) of implementation. For example, strategies from the “train and education stakeholders” and “develop stakeholder interrelationships” clusters were important in the first year while the strategies within the “use evaluative and iterative strategies” cluster were more closely associated with treatment starts in the second year. A more detailed accounting of the specific phases of implementation that were present over the reporting period could further clarify the relationships between phases of implementation and the strategies used. Our finding that the strategies associated with HCV treatment changed from Year 1 to Year 2 supports the notion that successful sites used evolving strategies as the clinical innovation became more available and as the learning collaborative evolved.

These results must be interpreted in the context of the national HIT Collaborative. The timing of the national efforts to improve HCV care also corresponded to a major shift in the treatment of HCV from difficult-to-use, interferon-based treatments to simple, highly efficacious, curative, pill-based treatments. We aimed to assess how the Collaborative influenced the choice of activities to promote HCV care and how much of the strategy uptake related to the HIT Collaborative itself versus independent, local activities in response to the availability of these newer HCV medications. We asked providers to comment on whether strategies would have been done without the Collaborative and assessed how members of the Collaborative employed strategies differently than non-members. We found that approximately half of all implementation efforts nationally were attributed to the HIT Collaborative, meaning that providers felt that half of the activities would not have been done without the HIT Collaborative. Moreover, the activities that members of the Collaborative were more likely to engage in were those that are considered to be core elements of learning/quality improvement collaboratives in the literature [3]. For example, education/training, team-building, and communication with leadership are all essential elements of learning collaboratives and were endorsed more frequently by HIT members than non-members. Thus, these analyses are useful in assessing the role of the learning collaborative in the uptake of the clinical innovation via implementation strategy uptake.

While learning collaboratives are increasingly popular, their effectiveness is relatively untested [3]. These data provide preliminary support for the effectiveness of learning collaboratives. For example, strategies that were difference-making in this sample were often those strategies considered to be core components of learning collaboratives including using data relay, training and education, creating new teams, facilitation/technical assistance, and stakeholder interrelationship strategies. Moreover, the ERIC strategy “facilitate the formation of groups of providers and foster a collaborative learning environment” which specifically refers to learning collaboratives was significantly associated with treatment in both years, suggesting that the learning collaborative itself was associated with increased treatment. The HIT Collaborative specifically focused on building stakeholder interrelationships and using principles of system redesign including rapid cyclic tests of change. Sites frequently endorsed these types of strategies and attributed their uptake to the Collaborative. Additionally, the HIT Collaborative attribution in the "evaluative and iterative" cluster, which was critical to the success in Year 2, increased substantially from Year 1 to 2, indicating that the Collaborative was instrumental to site-level success. Most of the strategies endorsed more often by HIT members vs. non-members were significantly associated with treatment starts. These data thus provide some preliminary support for learning collaboratives as an effective means of increasing the uptake of a clinical innovation or evidence-based practice.

Implementation strategies have historically been difficult to measure. Generally, tracking strategies has been completed by observation and not by self-reporting on a comprehensive list of strategies [16, 17]. While we previously reported on developing a survey using the definitions of implementation strategies from the ERIC study, it remained unclear whether these strategies would be understandably and reliably interpreted by non-implementation scientists. In both years of data collection, we found an association between clinical outcomes with specific implementation strategies. The second year of data collection further demonstrates that providers could interpret and answer questions about implementation strategies. First, there was adequate interrater reliability within sites in both years. Second, there was consistency across the years in that several of the strategies were associated with treatment starts. Third, the strategies that were individually associated with treatment starts were in some cases those strategies supported by implementation literature. For example, facilitation is a well-studied strategy and was associated with higher treatment rates in Year 1 [18‐22]. Fourth, we found that the providers were able to generally distinguish between similar strategies. The strategy clusters were designed to group similar strategies, and we did not find strong correlations between endorsement of specific strategies within a cluster. In fact, there was significant variation in endorsement of the strategies within clusters (where the most similar strategies are housed). These findings indicate that such surveys can be used to track implementation strategies across a wide range of provider types, education, and geographic locations.

This study has several notable limitations. First, we relied on year-end self-report of implementation and included only one response per site. We found that the results had face-validity, as outlined above, and there was adequate interrater reliability when we assessed the reports of sites with more than one respondent. However, future studies would benefit from directly observing site-level implementation or documenting the application of implementation strategies over the course of the reporting time period. Second, it is unclear if the self-reported attribution data related to increased awareness of the HIT Collaborative or social desirability, given that more of the respondents in Year 2 were HIT members. Theoretically, any strategy could have been “correctly” attributed to the Collaborative, since the Collaborative leadership team encouraged and supported any strategy that the sites felt would be effective. A third limitation is that we included limited contextual factors in our associations between implementation strategy use and clinical outcomes. However, participant characteristics were not significantly associated with strategy endorsement. Given the uniformity of structure within VA, this may be less important, but in applying these lessons to non-VHA sites, more contextual information may have to be collected. We were also unable to assess the timing, sequencing, or intensity of implementation strategies within each year. It may be that specific strategies need to be sequenced in a specific order within the year. While the simple listing of strategies allowed us to quickly collect data from across the country, these data do not detail how the strategies were operationalized by sites. Often the application of implementation strategies may vary broadly and lead to difficulties assessing which elements of a strategy are critical or difference-making [23]. We successfully tracked strategies across the years, which is a strength of these analyses. A final limitation was our limited choice of outcome measures. We considered focusing on the proportion of patients treated as our primary outcome but were concerned that sites with fewer patients would have an artificial advantage, since sites treating half of the patients would look the same, whether they had 20 or 2000 patients to treat. Our findings should be validated in other clinical contexts with other medical problems. Future work in this area could aim to address whether specific combinations of strategies are important or how to use these data to address low-performing sites. Future work could also assess the associations between the sites’ stage of implementation [24] and strategy utilization.

These findings collectively indicate that the strategies associated with the uptake of a clinical innovation change over time from “pre-implementation” strategies including training and education, interactive assistance, and developing stakeholder interrelationships to strategies that are evaluative and iterative and adapt to the context, which indicate a more mature phase of implementation. This research advances the field by providing support for the implementation strategies and clusters developed by the ERIC group. This project demonstrates the utility of deploying surveys of implementation strategies sequentially across the life of a national clinical program, which could provide guidance to other national initiatives.