Introduction

Genomic technologies have been described as both disruptive and transformative.1 The ability to sequence an entire human exome or genome provides vastly more information of potential relevance to a person’s current and future health than was possible previously. As the relationship between sequence variation and disease management becomes better understood, genomic medicine—the use of information from an individual’s genome in the diagnosis and management of their condition—will become increasingly relevant to a broad range of health practitioners. However, numerous barriers to the implementation of genomics into clinical practice exist. These span the realms of workforce (capacity and behaviour change), service design and support, evidence-based practice, reimbursement, e-health (health and biomedical information), and ethics and regulation.2,3,4 Resolving these barriers in order to successfully integrate genomics into routine health care and improve patient outcomes requires change across each health system. Yet, Ginsburg observed that “most health systems around the world are not fit to shepherd genome sciences into routine health care” and that decisions are often made on an ad hoc basis.5

The introduction of genomics into a health system is complex, as resolving the diverse barriers requires change across many of its interacting component and health systems are not generally amenable to change.6 Several approaches have been taken to address the challenges inherent in genomics. Large scale sequencing initiatives and discovery-based research projects address gaps in understanding the clinical significance of genetic variation.7, 8 Global collaborations aim to accelerate progress by fostering the sharing of genomic data.9 Increasingly, there is recognition of the importance of implementation research to identify strategies which assist the adoption of genomics by clinicians.3 Each effort contributes to an important body of knowledge but forethought and simultaneous change across multiple areas of health care are needed to ensure that a genomic test is available for clinical use (implementation) and that clinicians apply the test and its results in practice (adoption). Focussing predominantly on a single aspect—be it the testing laboratories, technology, a specific medical discipline or clinical problem—will not be sufficient to create the widespread change necessary to ensure that patients’ health is improved by using genomic information in their medical care.

The introduction of genomics will require targeted support and is unlikely to occur by diffusion.10 Governments and organisations considering the implementation of genomics need to design a high-level, holistic model for how genomics will be delivered in the future, tailored to their own unique context. Implementing genomics requires considerable investment in infrastructure, personnel, and change management. Getting it wrong is likely to both be extremely expensive and reduce the potential benefits. These risks can be diminished by conducting a demonstration project, whereby a model (e.g., for the delivery of genomics in practice) can be tested to evaluate its strengths and weaknesses.11 Importantly, in health care, demonstration projects simultaneously provide an opportunity to foster change and adoption amongst stakeholders, while evidence can be gathered on a potential model’s design, cost and impact. In the case of genomics, demonstration projects provide a framework with potential universal applicability, facilitating local solutions to the challenges faced globally.

The Melbourne Genomics Health Alliance (‘the Alliance’) has conducted a demonstration project as a first step towards integrating genomics into health care across ten independent organisations. The results12,13,14,15,16 are now proving indispensable in guiding implementation and adoption of genomics in practice and patient care in Australia. Here we present our experience designing and conducting this collaborative project and share our lessons learned in the hope it will assist others to navigate their own path into the era of genomic medicine.

The Melbourne Genomics Health Alliance

The Melbourne Genomics Health Alliance (‘Melbourne Genomics’), established as a collaboration in 2013, is tasked with integrating genomics into health care across the Australian State of Victoria—a population of approximately six million people.17 Its members are ten leading healthcare, research, academic and service organisations, encompassing five hospitals, five accredited molecular testing laboratories and four clinical genetic services. Patient care across these organisations spans tertiary/ quaternary public hospital services in paediatrics to geriatrics. Public hospital care in Australia is largely funded by the State governments and is available free of charge at the point of care to all residents. There is no provision for reimbursement of genetic or genomic testing through personal medical insurance cover. Limited funding is provided by State and Federal governments, but for many tests no reimbursement is available.18 Funding for investigations can be established by application to one of a number of schemes, each with different scope, opportunities and limitations. Currently, health services have access to funding for only a limited number of clinically indicated genomic sequencing tests, restricting substantive testing to those eligible for research studies or able to pay themselves.

The members of the alliance recognised the need to work together to optimise the use of genomic information in clinical practice and research across the Victorian State health care system. To succeed in this goal, clinicians, patients, clinical laboratory scientists and researchers need seamless, secure access to genomic data and information. The members agreed to establish a common genomic data management platform that enables the analysis, storage and use of clinically generated genomic data from multiple diagnostic laboratories and its availability at six health care organisations and to several research institutions. This is an unprecedented level of change and cooperation between independent hospitals, accredited testing laboratories and academia—all organisations having separate and unique governance structures.

Importantly, the Alliance recognises an individual organisation’s right to make their own business decisions (e.g., to provide an accredited testing laboratory service) while supporting collaboration to drive change and optimise the use of resources to improve patient care and research opportunity. This paper focusses on the areas of change that were addressed collaboratively, not those that were within the remit of individual organisations (e.g., accreditation of testing).

Melbourne Genomics demonstration project

Demonstration projects sit at the interface between policy, research and implementation. They test the practical application of knowledge or experience—such as a structural innovation (e.g,. technology) or non-structural innovation (e.g., health programme or policy)—in a chosen setting, preferably under real conditions. They are usually conducted for a limited time and in effect test a prototype or a ‘proof of concept’ model to provide information, but are not intended to be scaled up. By contrast, pilot projects are designed and delivered as a programme is expected to be conducted, and can be scaled up for full implementation.19 Evaluation determines a model’s feasibility (can the model be built?), and requirements (what does the model need to do?) and/or impact (can it make a difference?).

A demonstration project which can guide the broad implementation and adoption of genomics in a healthcare system needs to resolve the numerous barriers that occur across the whole health care system. It should provide evidence of clinical utility and cost-effectiveness to inform test reimbursement, clinical guidelines and medical decision-making. It needs to determine how genomic data can be ethically and robustly managed so the information is available across institutions for clinical care and research. It also provides an opportunity to foster change in clinical and organisational behaviours such that genomic information is incorporated in the practice of medical specialists.

The Melbourne Genomics demonstration project was designed to provide evidence for the cost-effective use of genomics in clinical care, identify acceptable and practical clinical and diagnostic information systems, policies and procedures, and upskill the workforce through hands-on experience. The chain of actions and intended outcomes (known as the ‘programme logic’20) of the demonstration project is shown in Fig. 1.

Fig. 1
figure 1

Demonstration programme logic. This diagram summarises the programme logic model, that is the relationship between the demonstration project’s activities, its outputs and the intended outcomes

Full size image

Conducting the demonstration project entailed establishing a ‘proof of concept model’ comprising governance, policies, procedures, infrastructure, and software applications (Table 1). Each component of the model was developed and agreed through a consensus decision-making process. Decisions were guided by placing the principles which underpin clinical practice ahead of research, technological or commercial imperatives and political drivers. Each aspect of the proof of concept model was evolved and evaluated within health care delivery (clinical and pathology services) by offering genomic sequencing to patients meeting specific criteria (see Patient Testing below). The broad steps involved in providing testing and using genomic data within the demonstration project are shown in Fig. 2.

Table 1 The demonstration project's ‘proof of concept model’
Full size table
Fig. 2
figure 2

Steps in the pathway for patient testing and use of the genomic data undertaken in the demonstration project

Full size image

Four broad groups of activities were central to the demonstration project: patient testing, evaluation, workforce development and e-health.

Patient testing

Clinicians offered patients whole exome sequencing in a routine clinic setting prospectively, in parallel with standard clinical investigations. Therefore, patients were eligible if they had not had previous molecular investigations. This design allows evaluation of genomic sequencing as an early or first tier investigation and complements retrospective studies of patients who have undergone considerable investigation before receiving a diagnosis.8, 21, 22 Analysis and reporting of variants were restricted to genes currently known to cause the relevant condition or affect its treatment. Patients with one of five diverse clinical indications (Table 2) were offered participation in the demonstration project, with pre- and post-test genetic counselling. The clinical conditions span adult and paediatric care; germline and somatic testing; new and well characterised genes; those with well-established molecular testing pathways and those without. The diversity of these ‘flagship conditions’ enabled us to evaluate when common procedures and systems can be used and when they must be refined for specific diseases, as well as the impact of providing genomic sequencing for different indications.

Table 2 Description of the Flagships
Full size table

Evaluation: evidence for application

To inform decision-making about how genomics is best implemented into health care, rigorous evaluation is needed. As with genetic services,23 the use of genomic sequencing in practice can be considered a ‘complex intervention’ by the Medical Research Council (MRC) definition24 as more than one group is affected (e.g., the patient and their blood relatives), there are numerous, varied outcomes, and its use requires some degree of tailoring. Complex interventions are commonly tested using experimental designs to determine the efficacy of an intervention. However, the structured nature of experimental designs does not allow insights into how adoption of the intervention can be supported. We applied a hybrid effectiveness—implementation design,25 which is better suited to demonstration projects. This design allowed us to identify when genomics is a cost-effective investigation (impact evaluation), as well as how it can best be offered (process evaluation), alongside activities designed to foster implementation (Fig. 1).

By providing genomic sequencing in parallel to usual care, we were able to directly and prospectively compare the rate of detection of mutations, the impact on patient care (i.e., the number of patients whose care changed in response to the test results and the nature of this change) and cost effectiveness of these two clinical pathways. In contrast to retrospective studies—where patients diagnosed through usual care are not considered in calculations of detection rate or cost effectiveness—this approach best reflects the ‘new world’ of genomic medicine where clinicians can order genomic sequencing at first presentation. Consequently, we found a diagnostic rate higher than 50% with genomic sequencing for children with features strongly suggestive of monogenic conditions (‘Childhood Syndromes’) and one third of those diagnosed receiving a change in care as a result.12, 14 This is consistent with other clinically ascertained infant cohorts in the US and Canada.26,27,28 Our work provides evidence to guide the timing of genomic sequencing relative to other investigations. For example, we have demonstrated that if whole exome sequencing was applied after exhaustive investigations of children under 2 years of age with features strongly suggestive of monogenic conditions, every additional diagnosis made by whole exome sequencing would cost US$6327. Strikingly, when whole exome sequencing could replace most investigations there was a saving per additional diagnosis of US$1702.13 Evaluation of this rigour is essential to support cases for funding and reimbursement. One avenue to obtain federally subsidised testing is through the Medicare Benefits Scheme29 and a submission for inclusion on this scheme has been made based on our data.

Equally importantly, this design provides evidence for when such testing is not warranted. Although whole exome sequencing yielded a diagnostic rate of 25% in patients investigated for hereditary colorectal cancer (hCRC), we found no improvement in comparison to usual care, as all patients standardly received a panel test (unpublished data). Our experience demonstrates the importance of gathering evidence for each specific clinical indication and consideration of health and health economic outcomes as well as diagnostic rate. Although the diagnostic yield for patients with focal epilepsy was relatively low (12.5%), one patient’s management and clinical outcome was profoundly improved.16 In contrast, 40% of patients with hereditary neuropathy gained a diagnosis15 but immediate changes in patient management as a result were limited (unpublished data).

As important as outcomes is the efficiency of the processes. In order to identify acceptable and practical policies and procedures, data was gathered from participating patients and the clinicians, diagnostic scientists and informaticians involved in the demonstration project. As well as enabling iterative improvements, the process evaluation has provided rich information about key pressure points, laying the basis for further implementation research and identification of systems which will support the use of genomics in clinical service delivery. These results will be reported separately.

Workforce development: fostering clinical adoption

Medical practitioners are concerned about being adequately prepared for the use of genomics in their practice. Christiansen and colleagues reported that contributing factors include a lack of knowledge and laboratory guidance, time pressures, and a lack of standards.30 While continuing medical education is a pathway for practitioners to overcome knowledge deficits,31 education alone is unlikely to be sufficient to facilitate appropriate use of genome-scale testing by medical specialists.32 Behaviours such as clinical decision-making are affected by an individual’s capability, opportunity and motivation to act;33 which are in turn influenced by both factors intrinsic to the practitioner and external factors such as service environment.34

Our strategy for fostering change was centred on the use of experiential learning, providing opportunity with the intention of building capability and motivation. The outcomes we sought were multifaceted: to build clinicians’ understanding of the complexities of genomic sequencing, to foster future adoption of evidence regarding the value of incorporating genomics by clinical disciplines, and to ensure that the systems and processes fully implemented in the future will be embraced by clinicians. Clinicians from ten clinical specialisations (Table 2) had the opportunity to gain “hands-on” experience of genomic sequencing in practice through patient testing. As well as managing testing and test results, they participated in multidisciplinary meetings to review results and determine clinical significance. Clinicians were responsible for the activities relating to patient testing for each condition, including the evaluation of impact. At the outset, clinicians were encouraged to engage in all decisions relating to design of the demonstration model and, as described above, were key informants in its evaluation. We found surprisingly few challenges to engaging clinicians in the various capacity building activities. Good clinical leadership assisted and the main barrier was logistics, for example scheduling result review meetings that suited clinicians from different hospitals. Anecdotally, any variation in engagement between medical speciality groups appeared to relate more to clinical utility of the testing than medical discipline.

The clinicians who participated in these activities work in tertiary and quaternary clinical settings, as this is where the immediate need for genomics is currently concentrated. Our strategy for fostering adoption is not scale-able to clinicians across the entire workforce, but it does yield a cohort of informed clinicians to lead change within their disciplines as testing becomes more widespread. Demonstration projects designed for adoption of genomics in secondary and primary care may be beneficial when genomic sequencing tests become available to practitioners in these settings.

E-health

A fundamental difference between whole genome/exome sequencing and testing a limited number of genes is the potential for reanalysis of the data in the future. Ideally, a person’s genomic information should be available to themselves and their health practitioners over their life course, enabling the underlying data to be re-analysed as analytic tools and evidence for its use in care improves. In fact, the American College of Medical Genetics and Genomics recommend that periodic re-evaluation of an individual’s genome is desirable.35

Establishing a platform, which enables the efficient management and use of clinically generated genomic information, however, is far from a simple task.36, 37 To avoid duplication, the members agreed to establish a single interconnected platform, which would provide functionality for laboratory genome analysis and systems, guide clinician decision-making at the point of ordering tests and interpreting the results, integrate with each organisation’s separate information systems, and support patient access to genomic information. Figure 3 shows the components of a shared platform for management of genomic data and information and its relationships with the data management systems of the members. Crucially, there needs to be a two-way flow of information between clinicians and laboratories, via the genomics platform and the usual clinical and laboratory information management systems (LIMSs) within each member organisation. Of course, the platform needs to be compliant with privacy and health data standards and have strong data governance—that is, clear definition of the way in which data is managed to ensure its availability, usability, integrity, and security. More information on the components are provided in Supplementary Material 1.

Fig. 3
figure 3

Conceptual diagram of the Melbourne Genomics Health Alliance shared information management platform for future implementation. The core of the common genomic data management platform is the Genomic Data Repository—a central place to store genomic sequence data. The Clinical Tools, Diagnostic Tools and Patient Tools are shared by the Alliance members and integrate with their own systems including EMR and LIMS. Further detail is in Supplementary Material 1

Full size image

The feasibility of this ambitious objective was tested by creating a proof of concept information management platform used by all laboratories for the demonstration project (Table 1 and Supplementary Material). The functionality of this was limited to the infrastructure and software applications necessary for bioinformatics analysis38 curation of variants and reporting, and storage of data relevant for reanalysis and research. Electronic medical records (EMRs) had not been implemented within the participating hospitals at the time, so integration could not be tested. Instead, the proof of concept variant curation database was linked with a federated health data platform, BioGrid (BioGrid.org.au). Akin to fostering change amongst clinicians, participation in this aspect of the demonstration project enabled clinical laboratory scientists to gain first-hand experience in using applications that provide genomic information, prior to providing whole exome sequencing as a diagnostic test. A myriad of ethical, legal and technical issues required resolution for laboratory analysis and making access to genomic data available to all researchers within the Alliance. This experience was invaluable, enabling better informed decisions to be made about the functionality needed for the long-term systems to be implemented. The detailed system design of the long-term systems was based on this work and undertaken after completion of the demonstration project. Further information is available from the corresponding author.

Challenges and lessons learnt

Leadership

The Melbourne Genomics Health Alliance has become a paradigm for successful collaboration in Australia. A key success factor was the role of the executives of each member organisation. The Alliance and its demonstration project was initiated by the executive leader of each of the founding Alliance member organisations. It was these leaders who agreed the vision and objectives, provided funding, and set expectations for stakeholders within their organisations. They clearly signalled that this was to be strongly collaborative in practice as well as intent when they funded the demonstration project equally, irrespective of the organisation’s turnover or scale of genomics activity. This message was clear to those in their organisations, though stakeholders appeared to be primarily motivated by the desire to ensure genomic sequencing was available to patients. There were no direct incentives (in the form of Key Performance Indicators, for example) placed on stakeholders within each organisation to succeed.

Collaboration and engagement

Earlier efforts by the Alliance members to collaborate at an organisational level had failed, in part because agreement could not be reached on how existing activities could be aligned. As a result, some stakeholders had concerns about hidden agendas, for example the consolidation of diagnostic laboratories. To ensure that the Alliance was a “coalition of the willing” it was explicitly stated that consolidation of laboratories was not an objective. Instead, a vision was developed for how genomics would be delivered in 5 years’ time (rather than by whom). This allowed stakeholders to make business decisions which aligned with the 5-year vision of their own volition without compromising their current work. The accredited laboratories are an exemplar of how this worked in reality. Each laboratory decided independently the extent to which they would continue to develop gene testing panels and/or put resources into exome sequencing. They also determined the extent of their involvement in generating the patient’s sequence and in interpreting and reporting the results for the demonstration project. Convergence between the members’ individual strategic development and the 5 year vision was actively fostered by ensuring that the governance of the demonstration project included stakeholders at a range of levels within each member’s organisational structure. This convergence was gradual and determined independently within each of the relevant organisations.

Multidisciplinary collaboration on the demonstration project greatly strengthened the outcomes. Stakeholders engaged were those who needed to incorporate genomics into their role (clinicians, diagnostic scientists), those with experience of genomic sequencing (laboratory researchers, bioinformaticians), those who would receive testing (patients, community advisors), evaluators and those with other technical expertise (health informaticians, software and database developers). Obtaining consensus within multidisciplinary groups was challenging; clinical care, diagnostics, information management and research have different and at times conflicting drivers. Institutional allegiances at various levels added another degree of complexity. However, we adopted a number of strategies to promote the achievement of consensus and this proved easier than anticipated. Firstly, the principles or criteria upon which a decision would be made were agreed before any options were considered. Table 3 lists the overarching principles that guided the programme. These were elaborated as relevant for individual decisions, such as the selection of a bioinformatic analysis platform. The focus on patient care provided little opportunity for the pursuit of self interest and the use of guiding principles promoted clarity and transparency of decision-making. Secondly, in general, we found the people closest to the “coal face” were best placed to make the assessment of the options against the set criteria and tended not to be parochial. Lastly, people seemed more willing to reach agreement on decisions where the outcome—a policy, guideline or software—would be subject to evaluation than they may have been for decisions with longer-lasting consequences.

Table 3 Principles for decision-making
Full size table

Agility and project management

Generally, clinicians and researchers were very familiar with the process of research, where the work to be undertaken to address a hypothesis and the allocation of funds to achieve this have been detailed in advance, in a successful grant application. In contrast, the demonstration project was funded as an implementation project. The methodology was more akin to an information technology implementation project, with the member organisations being the project sponsors (funders), the agreed scope of work being documented in a project brief and procedures and systems being developed agilely. This was initially difficult for stakeholders to understand. Some voiced concern that there was not a detailed end-to-end description of every step, while others were anxious that decisions had already been made without their input. Good governance, inclusiveness and transparency of decision-making were crucial in allaying these concerns. Another source of confusion was the necessary delineation between areas of work that were identified as collaborative and those that remained fully in the realm of the individual organisation (e.g., sequencing technology acquisition).

In order to build capability and activity within the Alliance organisations, all parts of the demonstration system were built and run by stakeholders, incorporating external expertise as required. To reduce the risk of not completing the demonstration project, a small, dedicated project team facilitated the collaborative process, and were responsible for ensuring that decision-making aligned with the objectives of the Alliance. The project team also assisted management of stakeholders’ competing demands on their time, ensuring rapid progress. However, in the context of a multi-institutional demonstration project, a project team can inadvertently reduce stakeholders’ sense of ownership, jeopardising future adoption. The members and stakeholders are the Alliance, not the project team: stakeholders and programme team all needed to be reminded of this at times.

In retrospect, our decision to focus resources on project delivery, without allocating more resources to communication within the Alliance, was a mistake. The use of a demonstration project was a radical paradigm shift for stakeholders’ and better and broader communication at the outset may have lessened some of the early confusion. Time and exposure may also have been necessary for stakeholders to fully engage.

Conclusion

We have shown that the framework of a demonstration project is well suited to testing a model for the application of genomics in clinical care and can build substantial momentum towards implementing genomics into our healthcare system. The conduct of the demonstration project itself fostered change across many disciplines, domains and organisations. As the complexities were even greater than envisaged at all levels and in all activities, it has created a strong foundation for the next phase of work—implementation of systems into the health systems. The State Government of Victoria and members have provided further funding (AUS$35M) to implement genomics over the next 4 years, based on evidence from the demonstration project. Melbourne Genomics subsequently became the blueprint for the development of a national initiative, namely: The Australian Genomics Health Alliance funded (AUD $25M) by the Federal Government’s National Health and MRC.

The proof of concept model we developed and tested may provide some usable insights and our suggestions for others are provided in Table 4. In 2013 Manolio and colleagues proposed a roadmap for the implementation at a single institution.39 Most of the steps they proposed were very pertinent to the path the Melbourne Genomics Health Alliance took, but the order of these steps was quite different. These differences were largely due to the ‘top down’ initiation of the work in Melbourne and both the scale (implementation across multiple institutions) and the multi-faceted nature of our programme.

Table 4 Suggestions for those considering a collaborative demonstration project in genomics
Full size table

The differing histories, drivers and health care systems mean that other organisations and countries are likely to have a different view of how genomics can be best implemented. These differences serve to emphasise the value of demonstration projects as a universally applicable framework for identifying how complex innovations such as genomics can most successfully be implemented in health care systems.

Change History

A correction to this article has been published and is linked from the HTML version of this article.