Comparative impact of two continuing education activities targeted at COPD educators on educational outcomes: protocol for a non-randomized controlled study using mixed methods

We will conduct a pragmatic non-randomized controlled study using explanatory sequential mixed methods [21]. From June 2016 to October 2017, moderate-to-large groups of ≥10 educators will participate in a lecture on TPE (comparison group). From November 2017, and thereafter, small groups of < 10 educators will be involved in a CE activity on TPE that will include a role-play simulation (experimental group). In educators, quantitative and qualitative measurements will be undertaken before and after the CE activity. Four months after the CE activity, educators will provide COPD patients with TPE in their work settings. Measurements will be undertaken in COPD patients before and after TPE.

Our study was registered in ClinicalTrials.gov (NCT02870998) on March 15, 2016.

Educators will be allocated to a CE activity on TPE that will include a role-playing simulation and involve a small group of attendees (experimental group) or to a lecture on TPE that will be presented to a moderate-to-large group of attendees (comparison group). Hence, the CE activities differ in their number of participants and learning format.

Both CE activities will be similar in regard to their content and learning objectives. The content of the CE activities will pertain to TPE and the general learning objective of both CE activities can be described as: to be able to deliver effective TPE interventions to COPD patients. The specific learning objectives are adapted from the Canadian Network for Respiratory Care’s National Certified Respiratory Educator Learning Objectives [22]. They are presented in Table 2.

The study coordinator (M.G.) will attend the CE activities to ensure adherence to intervention protocols.

Determinants of educational outcomes identified a priori will be measured (see Table 3). To characterize study participants, we will collect data on attendees’ jobs, undergraduate studies, the number of years of professional practice, and former participation in CE activities, using a standardized form. In COPD patients, we will measure disease severity that is classified by symptoms and disability [38], along with marital status, highest attained level of education, and annual family income, using items derived from the Quebec Survey on Cardiovascular Health (QSCH) [39].

Figure 2 illustrates the schedule for enrollment, interventions, and assessments. The measurement time points have been chosen in accordance with the Expanded Outcomes Framework for Planning and Assessing Continuing Medical Education Activities [15] (e.g. satisfaction and self-report of competence are measured immediately after a CE activity) and feasibility issues (e.g. time for institutional feasibility approval for performance and patient outcome assessments). As illustrated in Fig. 2a, educators will be enrolled and will be asked to complete baseline measurements on the morning of the CE activity (t _{-1, educators} ). Among educators, post-activity measurements will be undertaken immediately after the CE activity (t _{1, educators} ), for satisfaction and self-report of competence, at 1-month post-activity (t _{2, educators} ), for learning, and at 2-month post-activity (t _{3, educators} ), for performance. Interviews will be conducted among educators at (t ₄ , educators), after quantitative measurements (details on the interviews will be given in section “Interviews with educators” below).

Four months after the CE activity, newly trained educators will deliver TPE to COPD patients. In COPD patients, measurements will be undertaken prior to TPE, at (t _{-1, patients} ), and six months after TPE, at (t _{1, patients} ).

The size of our sample of educators was calculated on the basis of our fourth study hypothesis, that predicted that post-CE activity performance levels will be higher in the experimental group compared to the other [13, 14]. In a previous study, educators were found to have a score of 0.89 on the effective TPE rating scale [25]. We think that the educators of the comparison group will also have this score. We expect the educators of the experimental group to have a score of 2.00, that is considered as good on this scale [25]. Therefore, we calculated that a sample size of seven educators per group was required to detect a group difference of 1.11 point (standard deviation, or SD = 0.63; type II error = 0.20, or 80% power; type I error = 0.05; two-sided test).

The size of our sample of COPD patients was calculated on the basis of our fifth study hypothesis, that predicted that patient outcomes will be further enhanced in COPD individuals who will meet the newly trained COPD educators of the experimental group, compared to the other [13, 14]. We expect the COPD patients, who will meet the newly trained COPD educators of the comparison group, to experience a change at 6-month follow-up equivalent to − 2 points on the impact dimension of the SGRQ-C, similar to the patients who were allocated to a previous usual care intervention [40]. Consistent with previous results [40], we expect the COPD patients, who will meet the newly trained COPD educators of the experimental group, to make an improvement equivalent to + 9 points. Using the SAS generalized estimating equation macro for controlled clinical trials with repeated measurements on the same individuals [41], we estimated that a sample size of 94 patients per group was required to detect a between-group difference of 11 points (SD = 19; type II error = 0.20, or 80% power; type I error = 0.05; two-sided test).

To account for possible losses to follow-up, to ensure Gaussian distributions, and because it was considered as feasible to recruit ≈5 COPD patients per educator, we hope to recruit 25 educators per group and 125 COPD patients per group (≈5 patients per educator). The CE activities will be delivered on several occasions from June 2016, and thereafter, to achieve the required sample sizes of both educators and patients.

The CE activities will be organized on behalf of the Quebec Respiratory Health Education Network twice per year, either in Quebec City or Montreal, Quebec, Canada. Upon request, the Quebec Respiratory Health Education Network may decide to schedule additional CE activities outside these two cities. Educators, who will participate in the CE activities held from June 2016 to October 2017, will be allocated the comparison group, whereas those attending the activity after November 2017 will be assigned to the experimental group.

Data analysts will be blinded to interventions, along with study participants, who will not be aware that there will be two different CE activities.

Before either CE activity, the study coordinator (M.G.) will invite all educators to participate in the study and will be in charge of their recruitment. Four months after the CE activity, the newly trained educators will invite their COPD patients to participate in the study. Educators will communicate the names and contact information of the COPD patients who will accept the invitation to take part in the study to the study coordinator. Then, trained research assistants will be responsible for the recruitment of COPD patients. We plan to stop the recruitment when we will reach the targeted sample sizes of both educators and COPD patients.

Questionnaires measuring educators’ satisfaction, learning (baseline/post-activity knowledge), competence, and confounding factors will be self-administered. For satisfaction, baseline knowledge, pre−/post-activity self-report of competence, and confounding factors, data will be collected on the CE activity site. Data on post-activity knowledge will be collected on the educators’ work site. We will send reminders to educators who will not return the post-activity knowledge questionnaire.

To measure performance for each educator, we will videotape one TPE intervention delivered to a real COPD patient in each educator’s professional practice. The study coordinator will send the camera to the educator by mail. Upon reception of the camera, the educator will invite the first patient to whom he or she will deliver TPE to participate in the study. If the patient refuses, then the educator will ask the following patient, and so on. We will ask educators to place a GoPro HERO Session™ camera (GoPro, San Mateo, CA, USA) on their desk. The camera has the height of 38 mm, the width of 38 mm, and the depth of 36.4 mm. It starts recording when pushing a single button and captures up to a 2-h ultra-wide field of view video in .mp4 file format.

Data on COPD patient health status, resource utilization, and confounding factors will be collected via telephone interviews with trained research assistants. The SLCDC is an interviewer-administered instrument. Previous studies have shown that the SGRQ and the CAT produce similar results when interviewer-administered [42, 43]. The heiQ, along with the Hospital Anxiety and Depression Scale (HADS), used to measure two confounding factors, will be mailed to patients and self-administered at each of the patients’ homes. Both the heiQ and the HADS could be interviewer-administered, upon request, to avoid missing data.

We will perform independent double data entry for numeric data (satisfaction, self-report of competence, patient outcomes, confounding factors). Two individuals will independently rate attendees’ learning and performance. When attributing knowledge scores, independent raters will be blinded to the measurement time point. They will score videotapes in random order. We will calculate interrater reliability using an intra-class correlation coefficient [44, 45]. Consensus will resolve disagreements. If necessary, a third reviewer will be consulted. All data will be stored on a secure server to prevent unauthorized access and loss of participant data.

Data will be analyzed by intention-to-treat [46]. The experimental group will be compared to the comparison group for all statistical analyses. To compare participant characteristics, we will use χ² test for binary outcomes and t-test for continuous outcomes. In educators, we will compare post-CE activity satisfaction and performance scores using multivariate linear regression models [47]. We will compare, in educators, change from baseline knowledge scores and change from baseline self-report of competence scores, and, in COPD patients, change from baseline patient outcome scores or counts, using generalized linear or Poisson mixed models with appropriate interaction terms (group × time) [48]. We will assume a specified form of covariance structure among the two repeated measurements (knowledge: t _{2, educators} versus t _{-1, educators} ; competence: t _{1, educators} versus t _{-1, educators} ; COPD patient outcomes: t _{1, patients} versus t _{-1, patients} ). We will assume a specified form of covariance structure among the two repeated measurements (knowledge: t _{2, educators} versus t _{-1, educators} ; competence: t _{1, educators} versus t _{-1, educators} ; COPD patient outcomes: t _{1, patients} versus t _{-1, patients} ). Estimates and standard errors will be based on a restricted likelihood function given the observed data. Specifying an unstructured covariance matrix, we will handle possible missing values at follow-up [49]. Depending on the variable type, we will calculate differences in means and counts with corresponding 95% confidence intervals [47, 48]. Model assumptions will be assessed. Determinants of educational outcomes identified a priori will be included in statistical models if they result in a > 10% change in the differences in means and counts [50]. We will examine the residuals to assess goodness-of-fit [47, 48]. We also plan to perform independent variable transformations if model assumptions are not met [51] or to use ordinal or binomial regression models. We will use an up-to-date version of SAS (Cary, NC, USA) to conduct all statistical analyses and two-sided p-values with α ≤0.05 level of significance for all tests.

We will conduct telephone interviews with educators of both the experimental and comparison groups to further explain our quantitative results [21]. We will purposefully select educators for interviews, based, for instance, on their performance scores or their professional characteristics. Based on the levels of assessment of the Expanded Outcomes Framework for Planning and Assessing Continuing Medical Education Activities [15], we will develop a semi-structured interview guide. The guide will also question the educators about the organizational support that they receive in their work setting in regard to TPE intervention delivery.

We will audiotape the interviews, transcribe interviews verbatim, and randomly check selected extracts of transcripts to ensure that there are no mistakes [21]. We will import data to QSR NVivo 11 software® and conduct content analysis [52]. Two members of our team will independently read and code three interviews’ transcripts, using an inductive approach [52]. They will compare their coding and debrief in order to develop the first version of the codebook. Based on this first version, they will independently assign codes to the data of the other interviews. We will compare their codes, resolve disagreements by consensus, and refine the codebook. Codes will be sorted into categories and the categories into themes. We will conduct interviews until saturation is reached [21]. Findings from the final analysis will be presented to educators to determine their accuracy [21].

In the discussion section of the published articles, results will be integrated in order that qualitative findings help expand or explain the quantitative ones [21].

Previous pre/post studies [53‐55] have evaluated the impact of CE activities targeted at COPD educators on some, but not all educational outcomes, as defined by Moore et al. [15]. None of these previous studies evaluated educators’ organizational support in regard to TPE intervention delivery, even though the organizational support is a major factor to consider when assessing the impact of a CE activity [12]. To the best of our knowledge, this study is the first controlled mixed methods study to compare the impact of two CE activities on TPE in regard to five educational outcomes, while qualitatively documenting how educators are supported by their organization.

In contrast to previous studies [53‐55], our study will objectively assess educators’ learning, using pre- and post-tests of knowledge, as suggested by Moore et al. [15]. Our study will also evaluate educators’ performance levels in delivering high-quality TPE interventions, based on observation measures of performance in each of the educator’s professional practice.

We expect our study to have limitations. First, non-differential measurement errors might occur, because:

Second, because the study is a pragmatic one, we expect that there could be some heterogeneity between the TPE interventions that will be delivered by each of the educators’ professional practice. Because what is referred to TPE may differ from a setting to another, heterogeneity is likely to have an impact on changes from baseline patient outcomes. Nevertheless, interviews with attendees will help to describe the implementation of TPE programs better, and, in turn, to explain our study results better.

Finally, we will compare, in the present study, two CE activities that will include two independent components: (1) the number of attendees (< 10 versus ≥ 10) and (2) the learning format (active versus passive). In contrast to a factorial design, our study design will not allow us to understand the effect of each independent component upon the educational outcomes.

To the best of our knowledge, our study is the first controlled mixed methods study to compare the impact of two CE activities on TPE in regard to five educational outcomes. The experimental CE activity was designed to promote the achievement of higher-order cognitive processes and align its learning objectives, activities, and assessments. We believe this study will serve as a model for evaluating CE activities on TPE. Results from this study could increase COPD educators’ performance levels in delivering effective TPE interventions, and, in turn, COPD patient outcomes.