Sustainability in Health care by Allocating Resources Effectively (SHARE) 10: operationalising disinvestment in a conceptual framework for resource allocation

Theories are based on concepts or ideas that characterise a particular phenomenon and propositions or relationships that link the concepts [12]. No specific theories of disinvestment have been proposed, however resource allocation theory, prioritisation theories, and decision-making theories have been applied in disinvestment projects; examples are listed in Table 2 [13‐18].

Perhaps the most relevant to disinvestment is the theory of discontinuance, defined by Rogers in his discussion of the theory of diffusion as “a decision to reject an innovation after having previously adopted it” [19]. In their review of diffusion of innovations in health care, Greenhalgh et al. note the importance of research into discontinuance and the lack of studies in this area [20]. Hollingworth et al. propose a schema of health technology adoption and withdrawal which includes both discontinuance and disinvestment [21] and Niven et al. use the definition of discontinuance for the term ‘de-adoption’ in their review of low-value clinical practices [22].

Frameworks use concepts and relationships to provide a frame of reference, organise and focus thinking and assist interpretation. Frameworks are descriptive, tend to be high-level and can apply to a wide variety of situations [12, 23]. No frameworks for systematic, integrated, organisation-wide approaches to disinvestment were identified, however there are several frameworks for specific aspects of the disinvestment process. These are summarised by setting, aims, method of development and components in Table 3. Those applicable to the local healthcare setting are discussed in more detail under the relevant steps in the disinvestment process below.

Polisena and colleagues [24] identified three frameworks in their review of disinvestment projects: Health Technology Assessment (HTA) [25], Accountability for Reasonableness (A4R) [26] and Program Budgeting and Marginal Analysis (PBMA) [27]. To distinguish between evaluation of new TCPs and those in current practice, the term Health Technology Reassessment (HTR) has been introduced for methods aiming to identify potential targets for disinvestment [28, 29]. HTA and A4R are frameworks by definition and are valuable tools for decision-making; however, although their use may lead to disinvestment, they are not frameworks specifically for disinvestment. Like A4R and HTA, PBMA and other priority setting frameworks [30‐32] can play a key role in certain approaches to disinvestment, but do not address all potential aspects of the disinvestment process or all opportunities to drive change. However they would all integrate readily into a wider framework for disinvestment, as aspired to with the trialing of the Australian Medicare Benefits Schedule Review initiative [33]. Recently Elshaug et al. provided a comprehensive inventory of disinvestment policy and practice levers that could flow from HTA/HTR and other priority setting processes [34].

Sources of synthesised evidence such as HTAs, systematic reviews and evidence-based guidelines, can underpin disinvestment decisions in two ways. Firstly, the process of evidence synthesis can be undertaken reactively to address policy, management or clinical questions as they arise and inform the resultant decisions. Secondly, dissemination of the findings of published HTAs, systematic reviews or guidelines can be a proactive method of initiating decision-making to ensure policy and practice is consistent with the best available evidence.

The ‘Disinvestment framework to guide resource allocation decisions in health service delivery’ [16] and the ‘Guideline for Not Funding Health Technologies’ (GuNFT) [35] are examples of frameworks to identify and disinvest individual TCPs. They are very similar to the process outlined in the Workflow Diagram of the New Zealand National Health Committee for introduction of new and removal of old technologies [36]. All three are systematic, transparent and based on a series of steps to identify suitable TCPs, engage relevant stakeholders, make the appropriate decisions, implement and evaluate change.

The New Zealand National Health Committee also includes a framework for wider stakeholder engagement in their Business Plan [36].

Joshi and colleagues use both framework and model when referring to the outcome of their narrative review ‘Reassessment of Health Technologies: Obsolescence and Waste’ [37]. Based on the definitions used herein, it is classified as a framework. It includes the role of reassessment in the life cycle of a health technology and triggers, structures and outcomes for health technology reassessment and decommissioning.

Bhatia et al. present an ‘Integrative framework for measuring overuse’ as an evaluation tool to be implemented within initiatives that aim to reduce ‘low value care’ [38] and Barasa and colleagues propose a framework for evaluation of priority setting processes which considers both procedure aspects and outcomes in a range of contexts [39].

Conceptual frameworks developed in the SHARE Program for a range of purposes within the disinvestment process include potential settings and methods to integrate disinvestment into health service systems and processes [5], components in the resource allocation process [6], an evaluation framework and plan for the overall SHARE program [40] and an algorithm to facilitate decision-making for selecting projects from an evidence-based catalogue of potential opportunities for disinvestment [9]. An existing framework for evaluation and explication of implementation of an evidence-based innovation was adapted for use in disinvestment projects [9] and health information products and services [11].

Models are more precise and more prescriptive than frameworks. They are narrower in scope, the concepts are well defined and the relationships between them are specific. Models are representations of the real thing [12, 23].

The SHARE Program produced three models: integrating consumer values and preferences into decision-making for resource allocation in a local healthcare setting [7], exploring Sustainability in Health care by Allocating Resources Effectively in this context [8] and facilitating use of recently published synthesised evidence in organisational decision-making through an Evidence Dissemination Service [11]. These are summarised in Table 3. No other models for disinvestment were identified in the literature.

The framework is aimed at health service decision-makers considering disinvestment and resource allocation, and health service researchers and implementation scientists working in this context.

The setting for this initiative was Monash Health, a large health service network in Melbourne Australia operating within a state-allocated fixed-budget model of financing. We anticipate results of this work and elements of the framework to have broader applicability and transferability, including to fee-for-service environments.

Decision-making in healthcare is described at three levels: macro (national, state/provincial and regional), meso (institutional) and micro (individuals) [42, 43].

The proposed framework was developed for use in policy, management and/or clinical decision-making at the meso level. It was designed to be embedded within existing systems and processes where it can be responsive to local needs and priorities at the level of implementation; for example health service networks, individual facilities, departments, wards or committees.

The lack of standardised terminology is a barrier to development of systematic approaches to operationalise disinvestment [1]. To address this, definitions and key concepts underpinning the framework are made explicit. The proposed framework provides a common language for researchers and decision-makers within and between programs, institutions and health systems making it easier to build and share a body of knowledge.

There are multiple definitions for disinvestment in the literature based on a range of different concepts [1, 44]. Numerous alternative terms conveying the same concepts are also in common use. Disinvestment is focused on the use of ‘health technologies’ but there is also a range of definitions for this term. To compound the difficulties arising from multiple definitions, the terms ‘disinvestment’ and ‘health technologies’ are frequently used in one way by researchers and in another by health service decision-makers [1]. Definitions relevant to the local healthcare setting are provided in Table 4.

We use the term disinvestment in the broadest sense, ‘removal, reduction or restriction of any aspect of the health system for any reason’. This can be applied to products, devices and equipment; clinical practices and procedures; health services and programs; information technology and corporate systems. Unlike most of the research definitions for disinvestment, this version is not constrained by a specified purpose (eg withdrawing practices of low value), defined criteria (eg effectiveness or cost-effectiveness) or anticipated outcome (eg reallocation of resources) which do not address cessation or limitation of TCPs for other purposes, based on other criteria, for different outcomes, which are likely to arise in local health services [1].

In contrast, we define health technologies in the narrowest sense; as products, devices and equipment used to deliver health care (eg prostheses, implantable devices, vaccines, pharmaceuticals, surgical instruments, telehealth, interactive IT and diagnostic tools) which reflects common use by health service decision-makers and consumers [1]. Clinical practices, health programs and services, information technologies, support systems, and organisational and managerial systems are not included in this definition. Although contained in many research definitions, they are not included in general references to health technologies in the local healthcare setting [1].

The terms ‘principles’ and ‘criteria’ are often used interchangeably; definitions for use in this review are included in Table 4.

The proposed framework is underpinned by several key concepts (Table 5). While disinvestment is the aim, it is not considered in isolation but in the context of resource allocation, addressing the spectrum of decision-making covering investment in new, continuation of existing, and disinvestment from current activities. The focus of the framework is positive: optimising healthcare, improving health outcomes, using resources effectively and efficiently. The components of the framework are integrated within current systems and processes and within existing health improvement paradigms such as evidence-based practice (EBP), quality improvement (QI) and system redesign.

Many of the elements within the proposed framework should be self-evident and be applied routinely as good practice, making it unnecessary to stipulate their requirement. However strong and consistent messages in the literature confirm that they are not standard practice and authors felt the need to state that they should be made explicit. Incorporating them all into a detailed framework achieves this.

Another reason for including all the elements in detail is to address potential ethical dilemmas [1]. In some circumstances it may be difficult to accommodate the principles of beneficence and utilitarian justice; clinicians advocate for the best interests of individual patients but resource allocation aims for the greatest benefit for the most people [45‐47]. Similarly, arguments for equity may conflict with those for efficiency when the most efficient outcome is not the most equitable [48‐50]. A systematic, transparent approach acknowledging these issues may facilitate difficult discussions and create potential for some efficiency to be traded away for equity maintenance or gain.

Some elements may be more important than others in individual situations. However, because they are all defined in the framework, the decision to exclude or reduce the role of some elements in extenuating circumstances becomes explicit. This strengthens the process and empowers those who have previously participated in suboptimal decision-making due to lack of resources, hidden agendas or organisational politics [6, 51‐57].

The proposed framework is composed of three interconnected and interdependent components: 1) a program for organisation-wide decision-making, 2) projects to implement decisions and evaluate outcomes, and 3) research to understand and improve the program and project activities. Each component has a number of elements which are outlined in detail below.

The framework is primarily descriptive to enable application in a local healthcare service and allow adaptation, replication and testing. It was developed using both deductive and inductive methods. Although not based on a specific theory, it has potential to facilitate future theory development and/or testing. Specific characteristics of the framework and potential for its use are summarised in Table 6 using domains and criteria developed to assess the robustness and utility of proposed models and frameworks [12]. This assessment enables potential users to identify whether the framework will meet their aims and be applicable to their situation.

Forty-two principles were identified from the existing literature and the SHARE publications and grouped into eight categories that emerged from these findings: Boundaries, Ethics, Governance, Structures, Processes, Stakeholder involvement, Resources and Preconditions. These are presented in the framework as two groups (Fig. 2).

The first group have a hierarchical relationship depicted as a series of nested boxes. The whole program is defined by explicit boundaries, ethical principles underpin good governance, governance directs and controls structure, and structure enables and accommodates process. The decision-making settings, prompts and triggers all sit within the scaffold of these five categories.

The second group, represented as three vertical bars, are required across all of the other elements. For example, stakeholders need to be involved in defining the boundaries and establishing the ethical parameters and methods of governance; they should be included in the structures and processes and participate in the projects and research. Adequate and appropriate resources and the noted preconditions will be required to establish, maintain and improve all aspects of the framework.

The intersection of the two groups of principles also demonstrates that ethics, governance, structures and processes also apply to stakeholder engagement, resources and preconditions. For example, stakeholder engagement should be systematic and integrated, funding should be sourced ethically and influence should be transparent.

These principles and their relationships also apply to the project and research components.

Further details of the categories, full descriptions of individual principles, and related citations are outlined in Additional file 1.

Nine settings for decision-making are described in three categories: Decision-making infrastructure, Specific initiatives and Individual decision-makers.

While the framework is proposed for organisation-wide application, any of the nine settings could be considered individually. A framework for a single setting would be underpinned by the same principles, decisions would lead to projects with the same steps and research could be conducted on all elements.

Prompts and triggers are proposed to initiate and facilitate identification of disinvestment opportunities. Prompts are informal reminders or encouragement for thought or action and triggers are formal mechanisms that initiate or activate a reaction, process or chain of events (Table 4). The settings above provide opportunities to introduce systematic prompts and triggers to use evidence from research, data and stakeholder feedback to drive decision-making.

Prompts, triggers and potentially even mandatory requirements to consider disinvestment could be built into existing decision-making infrastructure [5, 37]. Using expenditure decisions as an example, prompts and triggers could be incorporated into meeting agendas of finance committees, budgeting processes, application forms, algorithms, protocols or checklists. Mandatory requirements to consider disinvestment could be implemented as specific directions within purchase orders, explicit decision-making criteria for committees, or steps in application processes that require authorisation. Additional examples of prompts and triggers at the organisational level are outlined in Table 8.

In specific initiatives to implement health service improvements, prompts and triggers to consider disinvestment could be introduced into project management templates or training programs for project management, change management, quality improvement processes, etc.

Prompts, triggers and mandatory requirements could also be used to guide the decisions of individual practitioners in clinical encounters; these could be included in local guidelines and protocols to steer practice away from unsafe, ineffective or inefficient use of TCPs.

The disinvestment process begins when opportunities for disinvestment are identified from the activities in the settings above. Eight steps in the disinvestment process were ascertained from existing frameworks [6, 16, 35, 36]: Identification of opportunities; Prioritisation (if required) and Decision-making; Development of a proposal; Implementation; Monitoring, Evaluation and Reporting; Reinvestment (if required); Dissemination and Diffusion; and Maintenance. Two additional elements are included: some projects may require development of local criteria for prioritisation and decision-making and projects that aim to reinvest will need to measure the resources released as part of the evaluation process.

The first two steps are part of the decision-making program, the following six are undertaken in projects arising from the decisions.

Potential disinvestment opportunities can be derived from all of the decision-making settings discussed above, either incidentally or systematically from prompts or triggers embedded in local systems and processes. However, at the health service level, it is more common for disinvestment opportunities to be identified through ad hoc proposals based on individual’s observations or local knowledge than through a systematic evidence-based approach [9, 21, 79, 80].

The sources of information noted in the literature that could be used in these settings to identify disinvestment opportunities include research, health service data, expert opinion and stakeholder consultation. While any one of these sources could identify a potential target for disinvestment, ideally information from all four would be combined in confirming the appropriateness of the choice [5]. Evidence from research would be considered in light of local data. For example, if a systematic review or HTA identified a more cost-effective intervention to one in current use, decision-makers could use local data to assess whether the burden of disease, volume of use, likely impact and potential cost of change warrant the required disinvestment activities. Similarly, evidence from local data would be enhanced by using the literature to identify best practice. For example, if an audit of prescribing rates of a high cost drug finds variation between departments, a review of the appropriate research would confirm whether the higher rate is overuse and should be reduced or the lower rate is underuse and should be increased. Expert opinion and stakeholder consultation add clarification and important perspectives to these decisions and may also reveal examples of inappropriate use of TCPs not identified by other methods. The SHARE Program used the SEAchange model [41], a formal evidence-based approach to change, to ensure that evidence from research and local data, experience and expertise of health service staff, and values and perspectives of consumers were considered at each step (Fig. 1) [3].

Priority setting exercises clearly include a prioritisation process, however initiatives that identify their disinvestment targets by other means may need a specific prioritisation process to choose between the options available. Methods and tools for systematic, transparent and equitable decision-making may be used if prioritisation is not required or to complement the prioritisation process.

Prioritisation tools primarily focus on characteristics intrinsic to the TCP; however additional criteria may influence the decision to proceed with a disinvestment project in the local healthcare setting [9]. These might be pragmatic features that enhance initiatives chosen specifically as pilot or demonstration projects, such as opportunities for ‘quick wins’, or factors that affect the outcome of a project, such as likelihood of success and sustainability or potential usefulness of the evaluation.

There is a huge range of potentially relevant criteria for resource allocation decisions. Most authors emphasise that a list of criteria should be developed with input from all stakeholders to meet the objectives of individual situations. The commonly cited basic requirements include clinical parameters such as safety and effectiveness, economic measures such as cost-effectiveness and affordability, and social factors such as local values and priorities. Additional criteria will depend on the setting and context. Methods and tools to assist in assessment of safety and effectiveness [133‐136] and use of economic measures [137‐139] are available. Similar resources for consumer and community engagement are addressed in Additional file 1.

Deciding between several alternatives is a complex process requiring consideration of multiple factors. Health service decision-makers are often not good at this, relying on heuristic or intuitive approaches which ignore potentially important information [140]. Methods such as burden of disease analyses, cost-effectiveness analyses and equity analyses focus on some but not all of the available information [140]. Multi-criteria decision analysis (MCDA) allows consideration of all factors simultaneously, and although used widely in other scientific disciplines, it has only been used in health care relatively recently [76, 140].

The Star model, a “socio-technical allocation of resources” based on MCDA and health economic theory, has been piloted successfully in two settings, revised and developed into a toolkit [141‐143]. MCDA is also the foundation of the Evidence and Value: Impact on DEcision Making (EVIDEM) framework, which is being investigated further through research conducted by the international EVIDEM Collaboration [144].

While the components of the A4R framework are included within several principles in the new framework, policy makers, managers and clinicians may also wish to use the A4R terminology specifically in their decision-making processes.

A4R is also the basis for the Systematic Tool for Evaluating Pharmaceutical Products for Public Funding Decisions (6-STEPPPs) [145] and A4R and MCDA have been combined in other decision-making applications [146, 147].

Lists of criteria for consideration in prioritisation and decision-making have been published for disinvestment [78, 82, 85, 109, 148], including many who have applied or adapted the criteria framework proposed by Elshaug et al. [72]; resource allocation [6, 149‐151]; and general decision-making [42]. A tool to analyse gaps in priority setting has also been developed [152].

Many health service decision-makers use a prioritisation matrix, but most of these are developed locally and based on simple spreadsheets or business case templates [9, 48, 55, 153]. This variety of tools makes it difficult to compare costs and outcomes more broadly and there is some scepticism amongst decision-makers about the lack of rigour, transparency and skills involved in their local programs [21, 48].

There are also software applications to facilitate PBMA and generic prioritisation processes [27, 154, 155].

Once a decision has been made that there is a need for change, a proposal to meet that need and implement the decision is developed. When the proposal is drafted, the time and other resources required to implement and evaluate it can be assessed to determine if the benefits outweigh the costs of the exercise and to inform planning.

The range of potential disinvestment activities is broad and disparate. A proposal to remove a drug from a hospital formulary is likely to be very different to a proposal to close down an inpatient facility. No specific methods and tools were identified for developing disinvestment proposals, but generic materials for developing the program theory or rationale and defining the program logic would be useful [156‐164], as would business case proformas and communication templates.

Proposals are more likely to be successful if they have certain favourable characteristics and new initiatives are more likely to be sustainable if there is appropriate availability and adequate provision of critical factors to achieve and maintain the proposed components and activities [20, 165‐167]. A checklist of the factors influencing likelihood of success and sustainability is available [8].

Some successes with national approaches to disinvestment have been reported and may have elements that are generalisable to local circumstances [72, 85, 102]. However in some circumstances national approaches are not applicable at state/provincial, regional or institutional levels; for example removing or refining indications for reimbursement for ‘low value’ TCPs in national fee-for-service schemes for doctors in private practice may not apply to doctors working in state funded hospitals.

As noted above, there are also many examples in the EBP and quality and safety literature of successful projects at local level to remove unsafe or ineffective TCPs which are not labelled as disinvestment.

Many articles about disinvestment do not address implementation at all and some note that there are difficulties related to implementation but offer no solutions. Of those that do consider implementation, many of the comments are principles, captured in the section above, or barriers and enablers, captured below.

One recommendation for successful implementation is that “we could create conditions that make it easy for people to avoid using low-value health care services” [128]. Environmental changes such as closing services, physically removing products from storerooms and work areas, and eliminating items from formularies and purchasing catalogues should achieve this aim and result in complete cessation. In addition, if providers or recipients of a TCP, program or service receive financial reimbursement, removal of funding is likely to reduce use considerably, although not necessarily completely [64, 72, 117, 168, 169]. But not all disinvestment decisions can be managed with structural changes.

The need for an implementation strategy for each disinvestment activity is widely acknowledged. One disinvestment guideline details eight generic steps in their Action Plan [35], the SHARE Program used the SEAchange model for evidence-based change [41] to implement disinvestment pilot projects and support services [9, 10], and a model for ‘de-adoption’ utilises the ‘Knowledge to Action’ framework [22, 170].

A range of approaches to facilitate implementation of disinvestment decisions has been proposed. These include communication and educational materials [58, 72, 78, 117, 121, 171]; financial incentives and pay-for-performance [59, 64, 72, 117, 168]; reinvestment of resources saved [29, 78, 80, 172]; clinical champions [48, 80]; clinical pharmacists to monitor and advise prescribers [68]; quality standards [59, 117]; professional standards, maintenance-of-certification activities and practice audit [117]; prompts through guidelines, protocols, clinical pathways and decision support systems [5, 58‐60, 72, 82, 168, 171]; requirements to report variations from mandatory guidelines [59, 72]; monitoring and reporting of outcomes [72, 78, 168]; public reporting of provider performance [59, 117, 168]; training and re-organisation of staffing and equipment [10, 78]; and “picking low hanging fruit” before tackling more difficult projects [80]. These proposals have arisen from qualitative work with stakeholders or been derived from an understanding of implementation science; the papers offering these suggestions for implementation do not report application or evaluation of these strategies in the disinvestment context.

Several authors note that implementation is more likely to be successful if decisions are made at the local level, integrated into everyday decision-making and central to local planning [55, 59, 60, 80]. A well-resourced and well-designed formal priority setting entity is reported to improve implementation of decisions [27, 37, 55, 173]. It provides a recognised vehicle to consider information such as new evidence or local performance concerns, one which has transparent processes and appropriate authority for decision-making and action, where local expertise can be built up and local knowledge utilised. It is thought to “make contentious decisions more palatable and defensible” [55].

The SHARE Program used the Technology/Clinical Practice Committee (TCPC) as a formal decision-making structure [2]. After piloting several approaches, the Evidence Dissemination Service mentioned above as a method of identification, was finally implemented within a governance model to ensure maximum adherence [11]. Recently-published, high-quality synthesised evidence was identified and publications reporting evidence of harm, lack of effect or findings of a more cost-effective alternative to current practice were prioritised for dissemination. An Evidence Bulletin summarising an individual publication was provided to the TCPC, which then forwarded it to the department head or committee chair responsible for practice in the specific topic area. A response was required to confirm whether current practice was consistent with the evidence, and if not, what measures were being taken to address this or an explanation of why change was not required. When there was evidence of harm, responses to the TCPC were required within 1 month and the responses, or lack thereof, were reported to the Chief Executive the following month. Responses to other Evidence Bulletins were required in three or 6 months. A total of 175 publications were disseminated, 55 of the Evidence Bulletins required responses. Of the 43 responses received during the evaluation period, 32 reported that local practice was consistent with the evidence, six reported that the evidence was not applicable at Monash Health, three noted that local practice was not consistent with the evidence but provided a justifiable reason, and two reported that remedial action was planned to bring local practice into line with the evidence [11].

Although there are some particular challenges to asking people to stop doing things they believe in [1], the general principles of implementation should apply to disinvestment as they do for any practice change. These are summarised in the SEAchange model and the Knowledge to Action framework: engaging all stakeholders, identifying what is already known about practice change in the topic area from the literature and local knowledge, undertaking an analysis of local barriers and enablers, developing an implementation plan including strategies to minimise barriers and build on enablers, piloting and revising as required, and finally implementing in full [41, 170].

The Schmidt ‘Framework for disinvestment’ notes that both process and outcome evaluations should be undertaken but provides no details [16]. In their framework for evaluation of priority setting processes, Barasa and colleagues propose measures for both procedure aspects and outcomes [39] and a systematic review summarises a range of performance measures to assess use of ‘low value’ TCPs [174]. The ‘Integrative framework for measuring overuse’ lists measurement tools linked to specific project/program goals and discusses advantages and disadvantages of each approach [38].

The SHARE Evaluation Framework and Plan was created for an organisation-wide program of disinvestment in a local health service network [40]. It was developed in consultation with stakeholders and included evaluation domains, audience, scope, evaluation questions, sources of data, methods of collection and analysis, reporting and timelines. A theoretical framework and taxonomy adapted for evaluation and explication of disinvestment projects was also used to understand the process of disinvestment in the SHARE Program [9].

The deficiencies in available economic and usage data and lack of methods for quantifying savings are considered to be significant limitations to evaluation [46, 60, 82, 175, 176].

There are many generic guidance documents for monitoring and evaluation of health programs and projects in a range of settings. Like implementation, the principles, methods and tools for evaluation should be as appropriate for disinvestment as they are for any healthcare improvement project.

Findings from monitoring and evaluation activities should be reported on a regular and/or scheduled basis to the appropriate stakeholders in accordance with project terms of reference, governance protocols and other local requirements.

This step will only apply in certain projects when it is anticipated that firstly resources will be released and secondly that they will be available for use elsewhere. Although there is considerable discussion about the potential for reinvestment or reallocation, there is little information about how to do it [1]. Resource release and reallocation are built into prioritisation processes for budget-setting but are not integral to other methods of disinvestment. One proposal for a “sensible, well-managed reinvestment program” describes “a cost-accounting process to capture, and a financial strategy and analysis to return, a pre-agreed portion of real savings” [172]. However the comments by other authors regarding inconsistencies in accounting practices, insufficient valid and reliable data, lack of methods and tools and absence of reported examples suggest that this may not be currently achievable [1, 21, 48, 60, 83, 175‐177].

These terms have been used with specific, but inconsistent, meanings in the disinvestment literature. For example, diffusion has been used to refer to uptake of ‘new’ technologies where disinvestment is used for removal of ‘old’ technologies [178]. In contrast, diffusion and discontinuation have been used to represent ‘spontaneous’ uptake and removal of technologies where dissemination and disinvestment are their counterparts for ‘managed’ uptake and removal [21]. The former links disinvestment with diffusion, the latter with dissemination.

Since the focus of this framework is on implementation of change, and does not differentiate between implementation of investment or disinvestment decisions, the definitions of dissemination and diffusion are taken from the knowledge translation literature (Table 4) [20, 170, 179]. Dissemination involves planned, active processes to share and spread information; diffusion is unplanned and passive.

Outcomes of disinvestment projects should be disseminated to others working in this area to fill gaps in knowledge, avoid duplication, build on successes and learn from mistakes and misfortune. However no guidance for systematic dissemination or facilitation of diffusion of successful disinvestment initiatives at the local health service level was identified. Guidance from the knowledge translation, EBP, QI and implementation science literature for dissemination and diffusion of new TCPs may be a useful starting point, however the specific challenges of disinvestment may influence the generalisability of these methods [1, 180].

Maintenance is the final step in the change process. It involves strategies to sustain recently implemented change after project support is removed; to integrate the change into organisational systems, processes and practices; and to attain long-term viability of the change (Table 4). Several terms are used in the broader health literature to capture this concept; examples include adoption, assimilation, sustainability and institutionalisation. Sustainability has been used in the context of disinvestment [3, 8, 22, 169, 181]. Maintenance is used in this framework to avoid confusion with use of the term ‘sustainability’ in a different context in the title of the SHARE Program. Maintenance is also used in the evaluation literature to assess “the extent to which a program becomes institutionalized or part of the routine organizational practices and policies” and can be applied to both the population targeted for behaviour change and the organisation that enacted or adopted the policy [182].

Montini and Graham propose that the disciplines of “Science and technology studies, the History and philosophy of science, the Sociology of health and illness, and Medical Anthropology” be explored to understand the factors relating to sustaining change related to ‘de-implementation’ [169]. Niven and colleagues recommend that ‘de-adoption’ interventions include a sustainability plan to prevent healthcare providers knowingly or unknowingly reverting to old practices [22].

The SHARE Program applied, adapted and developed methods and tools to facilitate sustainability of disinvestment-related initiatives at both the program and project level.

Although a rigorous systematic approach was taken, it is impossible to be comprehensive in ascertaining all the relevant literature on disinvestment; the reasons are outlined in the conceptual review [1]. As a result, some relevant publications may not have been identified and some information may not have been published. Despite these limitations, the messages about operationalising disinvestment are generally clear and consistent and provide strong underpinnings for the framework.

The literature has been reviewed from the perspective of a local health service, however the authors’ experience is based in the Australian health system; hence differences with other health systems may not have been recognised and additional decision-making settings or methods and tools may have been missed.

The specific details of the ‘where, who and how’ of decision-making is likely to differ between organisations but the underlying premises should be the same: individuals and groups make decisions under certain conditions. The classifications of decisions and decision-makers might be useful starting points to elucidate local particulars.

The proposed framework is conceptual and untested. Naming of categories, determination of their constituent elements and the relationships between components has not been piloted or refined with stakeholder input. It is large, complex and all-encompassing and may prove too daunting or complicated to be achieved in this format. Testing and research may establish if it is feasible in the current overarching format or if it should be renamed, redefined or reformulated for implementation in smaller sections.

The framework is proposed at the ‘big picture’ level and requires supplementation with detail for all the components. There are some existing frameworks, models, methods and tools that can be applied in several areas but not for all elements within the framework.

There are many barriers that cannot be addressed by generic system changes and must be tackled when implementing the framework in individual situations. Many of these may be successfully overcome with local strategies; however some aspects of the framework involve potentially insurmountable barriers in the current environment. The main example is lack of valid, reliable, timely, appropriate and sufficiently specific data to identify disinvestment targets and monitor and evaluate disinvestment initiatives.

As the focus of this review is operationalisation of disinvestment, the implications for policy and practice have been integrated throughout the paper.

The implications for research in operationalising disinvestment are enormous. Placing the research component of the proposed framework across all the constituent elements illustrates that there is a need for research in each of them. Some topics stand out as priorities.

Many authors report a lack of frameworks, models, methods or tools for disinvestment. However there are some frameworks and models for disinvestment, although not tested; and plenty of methods and tools, many of which are tested, frequently from other research disciplines but which are relevant for disinvestment projects. Perhaps a more important factor is the lack of proactive mechanisms, prompts and triggers [9, 11, 16, 21, 24, 27, 192]. There are rigorous methods for HTA and analysis of health service data but no systematic methods to initiate these processes or draw the results to the attention of health service decision-makers. It is also not clear who is, or should be, responsible for instigating and making decisions and taking action. Research in these areas is a priority.

Investigation of data requirements, data collection methods and skills of decision-makers to use data for disinvestment is another priority [21, 24, 27, 46, 48, 55, 58, 60, 131]. Support for data collection after a TCP has been introduced is low and research into methods and resources required for post-market surveillance and “coverage with evidence development” is required [24, 132].

Some authors have highlighted other aspects of disinvestment for research such as exploring disinvestment at local health service and individual practitioner level [16, 55, 56, 80, 188, 193], taking a longitudinal approach from inception through implementation to outcomes that cross organisational boundaries [80, 188], identifying determinants for disinvestment [15, 80, 129], implementing change management [56, 58], and drafting and refining frameworks, methods and tools [15, 24, 29, 58, 129, 175, 184, 185]. Mechanisms to develop, implement and evaluate disinvestment activities can be built on existing theoretical frameworks from other research paradigms such as HTA, knowledge translation and implementation science [28, 83]. Measures of impact, potential unintended consequences and factors contributing to success or failure also need to be captured [24, 83, 193]. The SHARE Program provides some early work to build on by reporting disinvestment projects from inception to implementation [9]; identifying determinants for disinvestment, potential unintended consequences and factors contributing to success or failure [9]; and developing frameworks, models and algorithms [5‐9, 11] and evaluation frameworks and plans [10, 11, 40]. These outputs of the SHARE Program are summarised in Paper 1 [3].

Research could also include testing the proposed framework in different contexts.