The ASTUTE Health study protocol: Deliberative stakeholder engagements to inform implementation approaches to healthcare disinvestment

As health system managers increasingly seek ways in which to improve quality of care and constrain resource use, the concept of health technology reassessment with potential for disinvestment has gained currency. Having existed for some time in the realms of business and manufacturing, the consideration of disinvestment practices within healthcare is relatively new, with antecedents in evidence-based medicine (EBM) and health technology assessment (HTA). Disinvestment seeks to improve health outcomes by evaluating existing health services, identifying those that do not provide safe, effective, or cost-effective care, and redirecting funding away from these services and towards those with superior safety, effectiveness, and cost-effectiveness profiles through a variety of policy approaches [1, 2]. It does not by definition need to be a dichotomous choice to fund or not to fund; disinvestment can occur by degrees, whereby subsidies may be restricted to subgroups of patients for whom there is evidence of potential for benefit based on specific clinical characteristics (referred to as ‘refining the indications’ for service provision).

While there has been international interest in disinvestment, it remains an essentially theoretical construction that has only been operationalized in a limited manner in few jurisdictions [3]. A number of challenges are faced in developing and implementing the concepts of disinvestment into pragmatic policy approaches. While methods for the assessment of new and emerging technologies and services have become relatively well-established in the processes of HTA, governments and other payers are still to determine the optimal processes by which to review existing technologies and procedures. Often these pre-date the EBM and HTA era and thus have escaped rigorous evaluation of safety, effectiveness, and cost-effectiveness, yet are entrenched as offerings within healthcare.

In considering processes for refining funding for established technologies and services, contested elements may include: what is counted as ‘evidence’ and by whom; the ‘burden’ of evidence required; and the consultative components that may be necessary to support disinvestment decisions. A best-practice model to support payers in pursuing explicit efforts towards disinvestment in a manner that may withstand challenge from vested interests and interested citizens is yet to emerge [1].

With a view to bridging this divide between policy and practice, a competitively funded, Australian-based research project was established in 2009. The purpose of the ASTUTE Health study (A ssessing S ervice and T echnology U se T o E nhance Health) is to develop, trial, and evaluate a transparent process by which to refine the indications for services, to ensure investment in procedures with favourable safety, effectiveness, and cost-effectiveness profiles. Independent of government, this multidisciplinary project seeks to incorporate methods from HTA and deliberative democratic theory and, as such, represents a novel approach to the development of evidence-based health policy wherein the views of both vested and non-vested stakeholders are seen to contribute valuably and uniquely to informing policy decisions within a disinvestment context.

Deliberative democratic methods have gained prominence in recent years, driven in part by the rise of evidence-informed policy-making and the potential for these methods to assist in involving the broader community in democratic governance that extends beyond voting [4, 5]. Current methods of policy development and decision making are considered to contain democratic deficits, created by the opaque, but often dominant roles of strong special interest groups/experts, an inability to represent heterogeneous public interests, and a lack of processes by which to encourage input from adequately informed members of the general public [6, 7]. Additionally, there remains a need for ‘health services to be accountable to users as taxpayers, voters, and consumers’ [8].

Debate around ‘the fair and reasonable allocation of resources has become a prominent one in modern day discourses about healthcare’ [9]. Health funding decisions, particularly those which relate to some level of reimbursement retraction (more effective redistribution notwithstanding) may carry significant moral implications, be highly contested (by a range of stakeholders) and historically have not been made in an explicit and transparent manner. The complexity and contestability inherent in disinvestment decisions position such health policies as key candidates for the integration of deliberative methods in their construction [4].

Within this study, members of the community will be involved in making explicit the barriers and facilitators of disinvestment (with reference to both the specific case studies and the broader disinvestment context), and to determine whether stakeholder engagement could contribute to creating improved and transparent decision-making processes in this policy domain. Additionally, it is anticipated that the colloquial evidence collected from community members will enhance the scientific evidence base derived from the systematic reviews, and balance the perspectives of other more actively vested stakeholders and special interest groups.

In order to align this research project with the realities of policy making, to which a wide variety of stakeholders contribute, the perspectives, knowledge, and interests of vested stakeholders (clinical and consumer groups) will also be included in addition to the views of non-partisan citizens. This broad, evidence-informed engagement seeks to mimic and enhance the diverse range of inputs with which health policy decision-makers must grapple in the course of negotiating health policy development; such inputs have not always been transparent. Greater accountability in decision making will be supported by the structured frame of discussion offered by a process of facilitated deliberation around an evidence base.

Health resource allocation decisions also may need to go beyond a narrow scientific view of evidence and consider ‘colloquial’ evidence [4] alongside scientific evidence. Deliberative and other consultative processes may usefully be employed to combine these different kinds of evidence to increase the likelihood of achieving ‘sound and acceptable decisions’ (Stern et al. in [4]). By including clinical and consumer groups in these processes, it is anticipated that critical experiential knowledge will be added to the scientific evidence collected through systematic review.

Herein, we describe the implementation of methods for broad stakeholder deliberative engagements, a frame for evaluation, and a costing of the process.

Two case studies were selected for the project: assisted reproductive technologies (in vitro fertilization (IVF) and intracytoplasmic sperm injection (ICSI) with specific reference to the safety, effectiveness, and cost-effectiveness related to female age, male age, and cycle rank); and vitamin B₁₂ and folate pathology tests. The services/procedures associated with both of these case studies are publicly subsidised under Australia’s universal health insurance scheme, Medicare.

The first phase of the project (see Figures 1 and 2) entailed systematic reviews for each case study. These reviews demonstrated the variable effectiveness profiles associated with both assisted reproductive technologies (ART) and B₁₂/folate tests [10, 11], and collated a wide range of evidence for stakeholders to consider, including ethical and costing analyses.

In the second phase, the findings of these reviews will be presented at a series of deliberative, evidence-informed stakeholder engagements designed to include both motivated stakeholders who have traditionally dominated health policy development (clinicians and consumers) and also the voice of the ‘average citizen’ which is often not heard or excluded from policy decision-making [7]. Each of the stakeholder groups will be actively facilitated towards identifying both shared and dissenting perspectives regarding public funding policy for each of the case studies in response to the evidence presented.

Finally, in phase three, policy advisors will be presented with the results of the reviews, alongside the perspectives of all the stakeholder groups collected in phase two. They will be invited to formulate a disinvestment opinion based on both the scientific and colloquial evidence presented to them, and then to explore their decision-making processes and rationale with reference to the disinvestment context.

The remainder of this paper will focus on project phases two and three as these represent the novel components of the process. A description of the systematic reviews and their findings has been published elsewhere [10, 11].

The two case studies of ART and pathology testing for vitamin B₁₂ and folate were considered to elucidate an understanding of the broader issues surrounding the development of disinvestment policy [12]. Both were identified as candidates for further assessment under disinvestment initiatives because they met multiple criteria on a proposed identification framework [1], including substantial temporal and geographic variation in usage and early evidence of differential effectiveness across patient subgroups. Additionally, the case studies were considered to engender a variety of different demands on a disinvestment process and were selected in part for their illumination of different potential responses from stakeholders. The ART case study was considered more likely to draw strongly on values-based arguments in the face of relatively low but rising volumes and high cost, while the vitamin B₁₂/folate testing case study represented an ostensibly non-controversial, low-cost service, albeit one with high and rising utilization.

To this end, the case studies were selected: to meet the dual goals of research and policy development; to create an innovative approach to shared decision-making; to evaluate the feasibility of participatory processes involving a wide range of stakeholders; and to learn about how these groups perceive disinvestment processes in the context of evidence-informed deliberation.

The deliberative engagements for each case study were designed and piloted by a multidisciplinary team (including clinical advisors). Ethics approval for all arms of this study was obtained from The University of Adelaide’s Human Research Ethics Committee.

The case studies were undertaken in a staggered progression to allow for refinement of the process with experience: ART was examined first, followed by the B₁₂/folate case study.

In order to avoid capture by vested interests, to maintain a power balance, and to cater for differing levels of expertise, stakeholders were consulted in separate engagements. Potentially partisan stakeholders were not mixed with lay citizens, but the ideas generated by each group were communicated to the subsequent groups to share knowledge and allow cumulative solutions to be constructed.

Methods of recruitment for the engagement sessions varied between the groups and are detailed in Table 1, with recruitment strategies constructed to take into account the objectives of each deliberative event alongside considerations of feasibility.

Participants in the clinical, consumer, and policy groups were purposively recruited on the basis of their experiential knowledge, stature as opinion-leaders, and role in decision-making processes. Nominations for clinical participants were sought from key stakeholder bodies in the respective fields of reproductive medicine and pathology. Nominees were then approached with a letter of invitation, as were key practitioners in each field.

Consumers (ART case study only) responded to an advertisement placed in the highest-circulating daily newspaper in South Australia. Due to the ubiquity of the test, an appropriate consumer group for the B₁₂/folate case study could not be identified.

Policy advisors across portfolios, divisions, and branches (state and federal) directly related to health funding were approached by the project’s Chief Investigators and invited to participate in the evaluative component of the project, with further policy participants identified through snowballing.

Recruitment of community members sought participants descriptively representative of Australian population demographics, although it is acknowledged that such a small sample cannot be completely statistically or politically representative [13]. Pre-determined stratification criteria (gender, age, household income) were applied by an independent recruitment company to a randomly sampled group of South Australian residents, thus avoiding oversampling from older or more economically-advantaged groups.

Sample sizes for the community engagements were based on those advocated for citizen’s juries [14‐17], while the other engagements were slightly smaller, reflecting the specificity of recruitment requirements. These were particularly rigorous for the ART case study, because the sensitivity of the topic under discussion required exclusion criteria designed to minimise any potential harm to participants. We felt a duty of care towards consumers who had not been able to achieve a successful live birth because this can be the source of much ongoing distress. The sample sizes were considered optimal to deliver a diversity of opinions while still allowing full and active participation in ‘communicative processes and will-formation’ [18]. From a research perspective, the sample sizes were sufficient to test the logistical and practical components of the engagement design [6], while also evaluating the benefits and drawbacks of the use of deliberative engagements in future decision making beyond the bounds of research.

The two case studies differed in their structure for a number of reasons. The ART case study engaged each stakeholder group in two ‘rounds’; this was designed to allow adequate time to address the morally and emotionally complex nature of the topic. It also allowed the opportunity for each group to hear the perspectives of the other stakeholders, and question, incorporate, or discard those views as they deliberated further.

Consumer engagements for the ART case study were approximately four hours long, and held on a mid-week evening over dinner. They included short evidence presentations and a wide-ranging participant deliberation. Sessions were designed to build from participants’ experiences, allowing them to extrapolate to consider broader implications for the healthcare system.

The process for the B₁₂/folate case study was streamlined, with general practitioners (equivalent to primary care physicians) the only group consulted twice. With the largest proportion of pathology test requests in Australia arising from primary care [19], it was considered critical to place significant emphasis on interaction with general practitioners in order to ascertain their views on the tests, the evidence, and the potential for change through the lens of primary care. The practicalities of assembling national-level stakeholders across multiple rounds for the purposes of the research precluded a second round of engagement.

The deliberative engagements for clinicians (specialists, general practitioners, and pathology providers) were designed to incorporate their significant experiential knowledge. Four-hour sessions, drawing on existing knowledge and experience of participants, included short evidence presentations and a meal, allowing approximately three hours of participant deliberation. The majority of these sessions were held in the state of South Australia, with only the pathology provider engagement taking place in another state (New South Wales; due to the participants being nominated by their national-level organisation).

The community engagements were held over two days (non-consecutive for ART; consecutive for the B₁₂/folate case study), designed to allow time for information sharing, deliberation, and relationship building, acknowledging that the participants were not content experts. Table 2 details the structure for each of the sessions, all of which included a variety of information presentation (adapted for the informational needs of each group), group activities, and deliberation. In all groups, deliberation occurred in a variety of formats, including large and small group brainstorming, small group discussion, and open discussion. Experts who had presented information (including a general practitioner, pathologist (B₁₂/folate case study), ethicist (ART case study), and research staff) were available for questioning on points requiring clarification or to further inform, but did not take part in deliberations.

In line with the evaluative nature of these engagements, phase three sessions with the policy makers will be structured around a presentation of the evidence base and discussion of the responses from stakeholders, followed by a set of open-ended questions that will allow them to explore the outcomes, strengths, weaknesses, and overall value of the process from a policy perspective. The two-hour roundtable sessions will be held in the Australian Capital Territory and South Australia, and commence with an overview of the project’s outcomes. Participants will be encouraged to engage in an open dialogue regarding each of the specific case studies and a reflection on the nature of disinvestment policy more broadly constructed.

For both of the case studies, the majority of information presented to the participants was based on systematic reviews of the relevant literature that were specifically commissioned as a component of this research [10, 11]. These were supplemented with information modules pertaining to other relevant aspects of health funding decision-making, including: description of the technology/procedure; an overview of health funding policy in Australia (complete with Medicare cost and utilisation data by location and demographics) and elsewhere; economic considerations [20]; and broad ethical frameworks to support decision-making [21] (see Figures 1 and 2).

Prior to the roundtable discussions, clinical participants were provided with a brief summary of the topic and a broad indication of the context for the discussion. At each session, expert members of the research group presented the information modules described above and remained available for questioning for the duration of the evening.

Participants in the consumer engagement (ART case study) were told the topic and context for discussion prior to the engagements, but were not provided with any detailed information. Evidentiary inputs were limited in the first round of engagement, which focussed on international variation in ART funding and previously proposed changes to ART funding in Australia. The second round of engagement included a brief presentation on age-specific safety and effectiveness, alongside the perspectives of other stakeholder groups and the funding models that they had constructed.

Based on multiple reports of lay citizens engaging successfully in technically complex areas of health policy [22‐24], this process of community engagement was designed around evidence-informed deliberations, requiring participants to be technically informed on a topic without influencing their deliberation [5]. For the community groups in each case study, the information presented was designed to be accessible to participants with no prior knowledge of the topic areas.

Aware that the topic of ART can engender highly emotional reactions, written versions of the information modules designed to be accessible to a lay audience were distributed one week prior to the engagement session to allow participants the chance to consider their views on the public funding of ART before entering into a potentially evocative group deliberation. This was not considered necessary for the B₁₂/folate case study due to its relatively more benign characteristics.

During these sessions there were a number of presentations from expert witnesses with regard to: current Australian health funding policies; safety and effectiveness parameters of the technologies; and economic [20] and ethical considerations [21] (see Table 2). Details of the clinical application of ART were also presented, and the B₁₂/folate case study included educative components related to the interpretation of test characteristics (e.g., diagnostic accuracy as measured by sensitivity and specificity) and clinical aspects of test ordering. The speakers who presented each of these sessions remained available to answer questions during subsequent deliberations.

Policy advisors will be provided with a detailed briefing paper one week prior to the session, providing an overview of the systematic review and summary points from each stakeholder engagement. Brief presentations will be provided by senior research staff, who will remain available to answer questions and provide clarification on any points for the remainder of the engagement session.

Clinical, community (citizen), and consumer (patient) engagement sessions were facilitated by an independent and neutral facilitator, who was not involved in participant deliberation or questioning of experts (except on points of clarity) and did not provide expert information. The facilitator managed the group interactions, ensuring that all participants had the opportunity to be heard; encouraged in-depth discussion and negotiation amongst the participants; and assisted participants in identifying areas of consensus and persistent disagreement in order to progress the discussion constructively.

Sessions with policy advisors will be facilitated by the project’s Chief Investigators, who will play a dual role in facilitating the evaluative discussion and providing information regarding the process and its outcomes.

Facilitators drew on micro- and meso-level research questions to inform the specific topics of case studies. At the beginning of each session and throughout the discussion, participants were asked to consider specific questions, with participants in the ART case study being asked, for example: ‘Should the criteria for public funding of IVF be changed? If yes, how and why? If no, why?’; and those participating in the B₁₂/folate case study: ‘What things should be considered when making decisions about how much we should publicly subsidise B₁₂/folate pathology tests?’ Meta-questions regarding the barriers and facilitators to disinvestment more generally and the role of various stakeholders in health policy decision-making were also explored with each group. Supplementary questions varied according to the participant’s role (clinician, consumer, citizen (community), policy advisor) and time was allowed for open, participant-driven discussion.

Members of the community, consumer, and clinical engagement groups were offered a small honorarium for each session they attended in order to cover any expenses associated with their participation. Policy advisors will not be offered an honorarium for their participation, because it is considered that consultations of this nature align with current government activity and thus fall within the remit of their professional roles.

Given the dual research/policy development objectives of this project, collection and dissemination of both the deliberative and analytical outputs of the stakeholder engagements were considered critical [22]. That is, our analysis is set to generate both a thematic representation of the content of deliberations, as well as more meta-level discussion around the ideological and material implications of the arguments and values articulated.

After obtaining consent from participants, deliberations were audio-recorded and transcribed verbatim by an experienced Hansard reporter (all speech machine typed in full as opposed to shorthand in real time). This occurred contemporaneously for all engagements.

The research component of the project demands that the content of participants’ deliberations be analysed in order to examine the social processes of negotiation and practical reasoning inherent in disinvestment decision-making. As such, the transcripts will be analysed using a synthetic approach to discourse analysis [25], combining the traditions of conversation analysis/ethnomethodology with an interest in broader cultural and historical power relations [26]. This approach allows a focus on the structure of participants’ accounts (how they were put together rhetorically so as to appear credible or persuasive in the deliberative context) as well as their function in justifying broader disinvestment outcomes.

Deliberative outputs of the process will be reviewed and discussed by the research staff with reference to the transcripts, in addition to the notes, lists, and other written outputs produced by the participants. Because pre-formulated options (requiring only a consensus ‘yes/no’ decision) were not offered to the participants, a singular, consensual ‘outcome’ of the deliberation may not be immediately (or ever) evident. For example, in preliminary analyses we see that participants revised their positions over the course of the discussions. This appears particularly evident in engagements with community groups, and requires that the report detailing the deliberative output of this group be endorsed by the participants after the conclusion of deliberation.

Table 3 details the expenses associated with the engagement components of each case study (completed at the time of writing). These costs may be variable, and will depend on the purpose, context, and extent of engagement. Although these costs may appear large at first sight, they are small compared to the cost implications of continued use (or otherwise) of the relevant technologies. The costs associated with undertaking the systematic reviews on which the stakeholder engagements are based are not included in this summary, nor is the time of lead study investigators.