Background
Aims
Research questions
Methods
Design
Participants
Data collection
Deliberation
Step 1: Identify the need for change
Needs assessment
Literature review
Step 2: Develop a proposal for change
Best practice guide
Likelihood of success and sustainability
Step 3: Implement the change
Barriers and enablers
Step 4: Evaluate the extent and results of the change
Evaluation
Ongoing quality improvement
Ethics
Results
Step 1: Identify the need for change
Needs assessment
Needs | Evidence of need |
---|---|
Identified at initial consultation | |
Appropriate and representative views in decision-making | Limited availability of committee members to attend meetings at short notice |
Sufficient preparation time for committee members to review applications | Applications provided to committee members 24–48 h before meeting |
Increased awareness of requirement for authorisation of new TCPs | New TCPs introduced in the organisation without application or authorisation |
More easily accessible application materials | Applicants expressed difficulty accessing application materials |
Explicit criteria for decision-making | Lack of documentation for how and why decisions were made |
Increased transparency in decision-making process | Lack of documentation of actual decisions |
Mechanism to appeal decisions | Applicants are unaware of recourse when they are unhappy with decision |
Resources to monitor newly introduced TCPs | Technology/Clinical Practice Committee (TCPC) run by Ethics Committee Secretariat without any additional resources |
Reporting of outcomes following introduction of new TCPs | No reporting structure or requirements |
Resources to develop, maintain, evaluate and improve rigorous systems and processes | TCPC run by Ethics Committee Secretariat without any additional resources |
Electronic communications to reduce inefficiency and inconsistency | All correspondence in hard copy |
Identified during program development | |
Appropriate categories of information about new TCP provided to decision-makers | Existing application form did not address all principles in Victorian Department of Health guidance |
Appropriate detail in information about new TCP provided to decision-makers | Existing application form allowed applicants to determine level of detail provided |
Issues of access and equity are considered | Not in previous Monash Health application form or Department of Health guidance |
Opportunities for disinvestment of current practice following introduction of new TCP are identified | Not in previous Monash Health application form or Department of Health guidance |
Standardised recommendations and conditions to capture and implement decisions | Not in previous Monash Health application form or Department of Health guidance |
Increased understanding, skills and resources in evidence based practice | Applications contained inappropriate information to establish evidence of effectiveness |
Availability of expertise in assessing costs and health service resource utilisation | Applications contained limited information about costs and resource use |
Process to assess when new TCP can be considered ‘standard’ practice, monitoring can be ceased and special patient information is no longer required. | New TCPs are introduced in a ‘probationary’ model. Outcomes are collected and reported and patients are informed that the TCP is new to the organisation and is being monitored. |
Process to assess ‘change in use’ of current TCP to identify any potential risks for the patient, clinician and organisation as a result of the change | Current use of TCPs may change to address a new indication or different patient population, if there has been modification to the equipment or technique, or if there are new operators or practitioners. |
Process to assess organisational issues (eg capacity, credentialing, funding) for research applications | HREC application process did not address these issues adequately |
Process for approval in urgent or emergency situations is in place | Not in previous Monash Health application form or Department of Health guidance |
Communication, collaboration and streamlining of processes between the Therapeutics, Technology/Clinical Practice, Human Research Ethics and Clinical Ethics Committees | Applicants submitting to one committee are often asked to submit to a second and sometimes third committee. This results in considerable delays in decision-making and requires additional documentation of the same information on different forms |
Patient information sheets are of high quality and consistent with Monash Health patient information format | Brochures submitted by applicants do not meet recognised standards of patient information, do not cover and are not consistent with Monash Health format |
Data collection is accurate and produced in a format that can be collated with others for monitoring and reporting | Many clinicians have no knowledge, skills or experience in data collection |
Literature review
Need for change
Step 2: Develop a proposal for change
Best practice guide
Program components
Program logic
Likelihood of success and sustainability
Success
|
Met
a
|
A proposal is more likely to be successful if it meets the following criteria | |
Based on sound evidence or expert consensus | ✓ |
▪ There is no clear evidence or recognised experts in the area of organisational decision-making for introduction of new TCPs | |
▪ However there is general consensus between the guidance documents and the local needs analysis, and no area of disagreement | |
Presented by credible organisation | ✓DM, ?App |
▪ National and state governments and a national professional body are seen as credible in this context by the decision-makers | |
▪ It is not known if all applicants will consider them credible as clinicians often see bureaucracy as intrusive and unnecessary | |
Able to be tested and adapted | ✓✓ |
▪ A formal pilot will be implemented and evaluated during the state health department funding round for new TCPs | |
▪ The whole program will be implemented in ‘pilot mode’, ongoing feedback will be sought and encouraged for the first 2 years | |
▪ The project team will adapt the systems, processes and resources based on the stakeholder feedback | |
Relative advantage is evident | ✓✓DM, ?App |
▪ The decision-makers value the benefits in improvements to transparency, accountability and use of evidence | |
▪ It is not known if all applicants will consider the changes to be an advantage over the previous system | |
Low complexity | ╳ |
▪ The process for introduction of new TCPs is complex and requires time, skills and expertise from the applicants | |
▪ The project team have made the application form as user-friendly as possible but it is still detailed and complicated | |
Compatible with status quo | ✓ |
▪ There are significant changes for both decision-makers and applicants | |
▪ However the changes make the new process very similar to Human Research and Ethics Committee applications | |
Attractive and accessible format | ╳, ✓ |
▪ The application form has been made as user-friendly as possible but is unlikely to be considered attractive | |
▪ Accessibility has been improved by creating a website as a single point of access for all information, documents and resources | |
Sustainability
| |
A proposal is more likely to be sustainable if it has appropriate and adequate provision in each category | |
Structure | ✓✓ |
▪ The Technology/Clinical Practice Committee (TCPC) is an appropriate body to manage this process | |
▪ Reporting to the Executive Management Team demonstrates that the process has a high priority within the organisation | |
▪ The roles of Chair, Executive Officer and Administrative Officer address all aspects of managing the process | |
▪ There is appropriate representation on the TCPC | |
Skills | ✓✓ |
▪ Members of the TCPC have skills in clinical practice, management, health service operations and finance, ethical and legal issues and evidence-based decision-making | |
▪ The Executive Officer and Administrative Officer have skills in managing and administering complex processes | |
▪ Staff with skills in finding, appraising and synthesising evidence; coding; analysing health service data; credentialing; business and finance; and infrastructure and equipment needs are available to assist applicants | |
Resources | ✓✓ |
▪ Website holds all information centrally | |
▪ Expertise is provided as noted above | |
▪ Online guidance to completing the evidence components of the application is provided | |
▪ Templates are provided to assist applicants and to ensure processes and documents are consistent and of high quality | |
▪ Adequate funding has been provided for the TCPC Secretariat to manage all seven components of the TCP Program | |
Commitment | ✓✓ |
▪ The health service has demonstrated commitment by making this process an organisational priority | |
▪ The Executive Director of Medical Services and Quality and Executive Director of Nursing and Midwifery are both on the TCPC | |
▪ All Program/Division Medical Directors, Executive Director of Nursing and General Manager of Allied Health are supportive | |
Leadership | ✓✓ |
▪ Members of the previous committee demonstrated leadership in striving to improve the process | |
▪ The TCPC is seeking to be a leader in introduction of TCPs by establishing a transparent, accountable, evidence-based process | |
▪ The Centre for Clinical Effectiveness is a leader in enabling evidence-based decision-making |
Step 3: Implement the change
Barriers and enablers
-
Economic and political context (n = 3): effect of state and national activities related to TCPs
-
Organisational context (n = 10): lack of time and resources, lack of awareness of current decision-making structures
-
Social context (n = 5): lack of influence, effect of perceptions and power relationships
-
Patient (n = 4): need for adequate consumer representation in decision-making, limitations in quality of patient information brochures
-
Individual professional (n = 17): lack of awareness, lack of knowledge and skills, poor compliance
-
Innovation (n = 6): complexity, time requirement, perceived lack of advantage
Implementation strategies
Pilot
Program introduction
Step 4: Evaluate the extent and results of the change
Evaluation
-
Recommendations from the Department of Health to other Victorian health services to use Monash Health methods and resources
-
Requests from several Victorian and interstate health services for permission to use Monash Health documents
-
Request from another state government to provide training in Monash Health methods for their state-wide decision-making body
-
Attainment of a national award: Australian Council of Healthcare Standards National Quality Improvement Award for Non-Clinical Service Delivery
-
Nomination for a state award: Victorian Public Healthcare Award for Doing It Better: providing sustainable, well managed and efficient health services
-
One specific element of the new program, the Joint Committee process, had a successful outcome that enabled an international breakthrough (Table 3: Case study) [18, 19]Table 3Case study: the Joint Committee processOne particularly significant innovation was the introduction of joint committee meetings for complex applications. Applicants submitting to the TCPC, Human Research Ethics, Clinical Ethics or Therapeutics Committees were often asked to submit to a second and sometimes third committee, depending on the nature, complexity and implications of their application. To reduce the duplication of paperwork, delays in decision-making and wasted time attending multiple meetings a collaborative process involving joint meetings and streamlined documentation was established. Details are outlined in the Toolkit [Additional file 1].The four committees held their first joint meeting to discuss Baby Z, a neonate with molybdenum cofactor deficiency, a rare metabolic disorder with no effective treatment that leads to death in early infancy. With the permission of Baby Z’s parents, the treating doctors sought authorisation to use a therapy that was effective in mice but had not been tested in humans.Members of each committee researched in their respective areas (eg scientific literature, legislation and regulations, preparation of therapeutic agents, etc.), provided documentation and contributed relevant expertise in the discussion. A report of the process and compilation of the documentation was undertaken by the Monash Health legal team. This information was used by the Victorian Office of the Public Advocate and the Family Court in their decision to allow the treatment. Baby Z survived [18].Clearly the scientists and clinicians deserve the credit for identifying and refining this ground-breaking treatment and diagnosing and treating Baby Z. However the decision to use an experimental treatment is a burden that should not be left to the treating clinicians and the family. The rigour of this transparent, accountable and evidence-based decision-making process utilising the specialist knowledge of relevant experts gave those involved confidence that this was the right thing to do. This information was subsequently accepted by other decision-making bodies around the world to expedite rapid treatment of the next few infants diagnosed with the same rare condition. Babies are now routinely treated at birth [19].
Ongoing quality improvement
Needs | Evidence of need |
---|---|
Governance | |
TCPC members should be of sufficient levels of seniority, credibility and influence to make and implement appropriate and acceptable decisions | Feedback suggested that all Medical Program/Division Directors and General Manager of Allied Health should be on the committee to own and drive decisions within their programs |
Expertise on infrastructure and equipment needs, contracts, maintenance, etc. should be available in the TCPC process | Applicants are corresponding with manufacturers/suppliers directly but do not have knowledge of contract negotiation, maintenance requirements, etc. |
Decision-making | |
Evidence provided should be based on a rigorous systematic review of the research literature | Applicants are not following guidance to undertake systematic reviews properly and not using templates in application form correctly. CCE frequently identifies existing systematic reviews or other evidence that has not been included by applicant. |
Application process | |
Independent experts should identify the best available evidence from the research literature | ▪ Applicant feedback is that they do not have the time, knowledge and skills to do this properly ▪ Applicants do not seek help from experts as required/recommended ▪ Information provided is incomplete, inadequate and/or incorrect ▪ Lack of objectivity results in overestimates of outcomes, underestimates of costs |
Independent experts should identify issues relating to resources (financial, space, equipment, staff) | |
Expression of Interest form should replace current application form | ▪ Applicant feedback is that form is not user friendly ▪ Project team observation is that form is not used correctly |
Business Case template should compare new TCP and current practice ‘head-to-head’ | Current process uses different methods to assess costs and resource utilisation for new and current |
Monitoring and reporting | |
Ethics approval as a Quality Assurance activity is obtained prior to data collection | Audits of patient information should be covered by ethics approval |
Resources | |
Data collection tool and Report proforma for diagnostic tests is available | Current Data collection tool and Report proforma are based on Department of Health requirements and only apply to therapeutic interventions. They do not support reporting of diagnostic tests. |
Sufficient staffing levels are provided for expert and independent input to application process | Relevant staff have full workloads and cannot add this unless other work is re-directed |
Program revision
Step 1 (revision): Identify the need for change
Needs assessment
Literature review
Need for change
Step 2 (revision): Develop a proposal for change
Best practice guide
Amendments to program components
Revised application process
Discussion
Limitations
What worked?
-
Use of an evidence-based approach to change guided by information from the literature, local experts and consumers
-
Multidisciplinary stakeholder involvement, including consumers, in development, implementation and evaluation of the project and representation in the systems and processes of the ongoing program
-
Assessment of barriers and enablers, characteristics of success and factors for sustainability followed by tailoring of strategies to maximise the benefits and minimise the problems identified
-
Implementation in a long-term ‘piloting’ mode that captured and acted upon user feedback for continuous improvement
-
Credibility of a program underpinned by international best practice
-
Provision of sufficient resources to undertake the project and deliver the program
-
Commitment, support and leadership from the Board, Executive, Senior Management and Clinical Directors
-
Skills of the CCE team in Evidence-Based Practice, knowledge brokerage and implementation of change