This paper describes the protocol of a mixed methods observational study to gain more insight into the concept of sustainability and its determinants. Two healthcare innovations were implemented in a selection of Dutch hospitals three to five years ago, using different implementation strategies. A Short Stay Programme was implemented in breast cancer surgery in 4 early adopter hospitals by means of a hospital tailored implementation strategy, and the Enhanced Recovery After Surgery programme in colonic surgery was implemented in 33 hospitals by means of a generic implementation strategy. As a result of these efforts, both cases showed improved short-term results in terms of hospital length of stay. Currently, it is unclear to what extent these innovative programmes and their results have been sustained. In the current study, the extent of consolidation of the implemented programmes will be assessed in fourteen hospitals in the Netherlands; four hospitals will participate in the breast cancer surgery case and ten hospitals will participate in the colonic surgery case.
Strengths and limitations of the study
This study will specifically and systematically address the sustainability of two healthcare innovations three to five years following two different implementation strategies. Regarding the breast cancer case, research on sustainability is not limited to long term effects of the implementation strategy, but also comprises policy cost-effectiveness. In the primary study, a policy cost effectiveness analysis on the Short Stay Programme was performed. As a consequence of e.g. changes in programme elements or case-mix, the current cost-effectiveness may be different. A recalculation of the cost-effectiveness will be conducted which will partly be based on the detailed cost- and effectiveness data collected during the primary study.
Another strength is the possibility to explore perceived determinants of sustainability from two different implementation strategies. Both programmes have been implemented in a multidisciplinary hospital setting. By studying these two different programmes, a large pool of medical specialists as well as other professionals can be approached. As such, this study will offer a multi-disciplinary perspective on sustainability and its determinants. We have chosen for the Consolidated Framework Implementation Research (CFIR) model to examine the determinants and the concept of sustainability. This framework is based on several scientific theories and existing models and considers a wide scope of possible determinants of sustainability.
In this study a mixed method approach will be applied, combining quantitative data with qualitative data. In the quantitative analyses we will examine whether the implementation results have been sustained, while in the qualitative analyses we will examine the professionals view on possible determinants of sustainability. The determinants of sustainability will be explored by means of semi-structured interviews with stakeholders of the healthcare processes and external change agents. This study is not aimed to analyse independent determinants in a quantitative manner by means of a multivariate regression model because we currently lack well-defined hypotheses on what are the most important determinants of sustainability. In this study we therefore also chose for qualitative methods.
Furthermore, the outcome and care process data will be extracted from patient files. The practical advantage is that the participating professionals are not claimed for intensive data collection by means of completing case record forms (CRFs). Also, by auditing the files of the last 40 breast cancer patients operated, and of the last 30 colonic surgery patients, a Hawthorne effect will be prevented. This is the phenomenon that a team is improving its performance due to awareness of being monitored.
This study has also some limitations. The generalisability of our quantitative results regarding sustainability will be limited due to the hospital selection. An analysis of sustainability of the ERAS programme in colonic surgery including all 33 participating implementation hospitals would have been optimal. Although the ten hospitals were selected based on carefully chosen criteria, it cannot be ruled out that some of the other hospitals that initially were not successful in achieving the ERAS goals, continued their activities and ultimately implemented the protocol successfully. The evaluation of the effects of the Short Stay Programme in breast cancer surgery after implementation will be conducted among the same hospitals as during the primary study. However, these early adopter hospitals may not fully represent Dutch practice regarding breast cancer surgery.
The analysis of sustainability is based on extracting data from existing data files. Although a Hawthorne effect is thus prevented, a potential limitation is that some data of interest for this study may not be properly recorded.
Not much is known about long term effects of implementation strategies and the determinants of sustainability. In this study, we perform an analysis on sustainability and we will try to shed light on potentially relevant attributes of sustainable change. We will build on previous work and aim to expand the knowledge regarding sustainability and its determinants. The results of this study will be relevant for researchers, implementation experts, healthcare practitioners and policy makers when making decisions about implementation and sustainability resource allocation.