Background
Innovation in continuing education: the ECHO model
Description and conceptual representation of the ECHO Model
Current knowledge gaps about the ECHO Model and importance of the proposed mixed methods systematic review
Aim and review questions
-
What is the effectiveness of the ECHO Model on healthcare professionals and patients’ outcomes? (QUAN question)
-
What are the experiences/views of ECHO’s participants, including both mentees and mentors, about what influences the development of competencies in healthcare professionals? (QUAL question)
-
What can be inferred from the QUAL synthesis on the experiences/views of ECHO’s participants that can explain the ECHO Model effectiveness in developing healthcare professionals’ competencies? (Mixed methods [MM] question)
Methods
Approach
Eligibility criteria
Participants
Intervention
Comparator
Outcomes
-
Competencies: Studies assessing changes in healthcare professionals’ competencies based on their participation in ECHO will be included in the QUAN component of this review. In the context of CE assessment, Moore and colleagues [55] have operationalized the concept of competence as “the degree to which participants show how to do what the CE activity intended them to be able to do” (p.3), which can be assessed using objective or subjective outcome measures. According to this definition, we will be including in this review both studies that report on the development of formal competencies based on their use in clinical or educational settings (i.e., simulation, observation), and studies reporting on participants’ self-reported competencies. We will consider for inclusion the following proxy outcome measurements for self-reported competencies: perceived confidence, self-efficacy, self-reported abilities, and intention to change [55].
-
Reaction (Kirkpatrick’s level 1): Healthcare professionals’ views and reactions to ECHO including outcome measures of satisfaction and participation (e.g., number of online participants, degree of retention, and attrition rate).
-
Knowledge (Kirkpatrick’s level 2): Healthcare professionals’ knowledge acquisitions based on their participation in ECHO, including objective (i.e., knowledge test) and subjective (i.e., self-reported knowledge) outcome measures of knowledge.
-
Behavior (Kirkpatrick’s level 3): Application of healthcare professionals’ new knowledge acquisitions in clinical practice based on their participation in ECHO, including objective outcome measures (e.g., interventions or tool utilization, initiation of treatment, patient charts, care performance indicators from administrative database) or subjective outcome measures (e.g., perceived clinical performance, self-reported change in care plan) of clinical performance.
-
Results (Kirkpatrick’s level 4): Changes in patients’ health due to changes in the practice behavior of healthcare professionals participating in ECHO, including objective outcome measures (e.g., measures recorded in patient charts or administrative databases and access to care) or subjective outcome measures (e.g., measures from patient self-reports) of health indicators. The health indicators targeted for this review will include outcomes such as health behaviors, health status, and well-being, including physical and psychological health, social functioning, and treatment outcomes.
Population
Phenomenon of interest
Context
Type of studies
Search strategy for identification of studies
-
Cumulative Index to Nursing and Allied Health Literature (CINAHL COMPLETE), via EBSCO
-
Evidence Based Medicine Reviews (All EBM Reviews), via OVID
-
Excerpta Medical Database (EMBASE), via OVID
-
MEDLINE, via OVID
-
American Psychological Association PsycINFO (APA PsycINFO), via OVID
-
Education Resources Information Center (ERIC, public access) (https://eric.ed.gov)
Study selection, appraisal, and data extraction
Study selection
Methodological quality assessment
Data extraction and management
-
First and corresponding author(s) information, publication year, and country
-
Study funding source(s)
-
Study objective(s) and design
-
Study population and health care setting
-
Time of study, method(s) of data collection
-
Planned and actual sample sizes
-
Participation and response rate
-
Results of significance to the QUAN review question (outcomes measures of competencies and levels 1 to 4 of Kirkpatrick’s model), including details on outcomes (definition, time points measured, missing data) and measurement (name of tool, measurement units, scales)
Synthesis and integration of QUAN and QUAL findings
Review question | Input | Technique | Output |
---|---|---|---|
What is the effectiveness of the ECHO Model on healthcare professionals and patients’ outcomes? (QUAN question) | Informal evidence: ECHO programs’ components/characteristics (from QUAN studies and QUAN component of MM studies) QUAN results: Critical outcome: Competency development Important outcomes: Corresponding to Kirkpatrick’s four levels of program evaluation (reaction, knowledge, behavior, and results) | Descriptive synthesis and meta-analysis (QUAN synthesis) | - Description of ECHO programs’ components/characteristics - Effect size measures of ECHO programs on critical and important outcomes |
What are the experiences/views of ECHO’s participants, including both mentees and mentors, about what influences the development of competencies in healthcare professionals? (QUAL question) | Informal evidence: ECHO programs’ components/characteristics (from QUAL studies and QUAL component of MM studies) QUAL results: Experiences/views of ECHO’s participants about what influences the development of their competencies (from QUAL studies and QUAL component of MM studies) | Descriptive and thematic synthesis (QUAL synthesis) | - Description of ECHO programs’ components/characteristics - Themes relating to what influences the development of competencies in healthcare professionals participating in ECHO |
What can be inferred from the QUAL synthesis on the experiences/views of ECHO’s participants that can explain the ECHO Model effectiveness in developing healthcare professionals’ competencies? (MM question) | Resultant QUAN and QUAL findings (from QUAN, QUAL and MM studies) | Juxtaposition of findings using a matrix approach (MM synthesis) for the purpose of exploring the correspondence between the ECHO Model effectiveness (QUAN synthesis) and the experiences/views about what influences the development of competencies in healthcare professionals (that we aim to gather from the QUAL synthesis) | MM inferences providing an overall interpretation of what is associated with the ECHO Model success (or failure) in developing healthcare professionals’ competencies |
First step: Descriptive synthesis
Second step: QUAN synthesis
-
Intervention: topic(s) or health condition(s) targeted in the program
-
Population: professional group(s) participating in the program
-
Context: practice setting of participating healthcare professionals
-
Study design: QUAN randomized vs. QUAN non-randomized controlled studies