Introduction
Methodological Gaps in the Study of EBP Implementation Preferences
The Current Study
Method
Participants
Attribute Development and Survey Design
Attribute | Latent class segment | Wald | |||
---|---|---|---|---|---|
Segment 1 | Segment 2 | ||||
Attribute levels | U | Z | U | Z | |
Social context | |||||
Supervisor support for EBP | 8.24a
| ||||
My supervisor does not support this EBP | −0.97 | −4.11 | −1.42 | −16.54 | |
My supervisor supports this EBP 33% | −0.02 | −0.12 | −0.19 | −3.29 | |
My supervisor supports this EBP 67% | 0.12 | 0.78 | 0.59 | 10.47 | |
My supervisor supports this EBP 100% |
0.87
|
5.64
|
1.02
|
18.17
| |
Colleague support for EBP | 14.52b
| ||||
0% of my colleague support this EBP | −0.81 | −4.08 | −1.59 | −19.28 | |
33% of my colleagues support this EBP | −0.37 | −2.18 | 0.09 | 1.56 | |
67% of my colleagues support this EBP | 0.53 | 3.82 | 0.70 | 12.94 | |
100% of my colleagues support this EBP |
0.65
|
4.37
|
0.80
|
14.82
| |
Trainers expertise and engagingness | 32.13c
| ||||
Trainer is not engaging nor an expert | −0.57 | −3.04 | −1.71 | −18.37 | |
Trainer is engaging but not an expert | −0.09 | −0.55 | 0.02 | 0.27 | |
Trainer is an expert but not engaging | −0.06 | −0.39 | 0.01 | 0.11 | |
Trainer is an engaging expert |
0.72
|
3.50
|
1.68
|
26.60
| |
Evidence of effectiveness | |||||
Percent of clients benefiting | 7.17 | ||||
Would benefit 25% of clients | −0.92 | −4.26 | −1.49 | −18.04 | |
Would benefit 50% of clients | −0.49 | −2.64 | −0.24 | −4.03 | |
Would benefit 75% of clients | 0.57 | 4.05 | 0.57 | 10.57 | |
Would benefit 100% of clients |
0.84
|
4.40
|
1.17
|
19.52
| |
Effectiveness in other agencies | 23.59c
| ||||
This EBP is proven in research settings, but untested in agencies | −0.58 | −3.01 | −1.22 | −15.60 | |
This EBP is proven in research settings and 1 agency | 0.02 | 0.15 | −0.32 | −5.22 | |
This EBP is proven in research settings and 5 agencies | 0.16 | 0.99 | 0.57 | 10.36 | |
This EBP is proven in research settings and 10 agencies |
0.40
|
2.93
|
0.96
|
17.31
| |
Organizational fit of EBP | |||||
Modifiability of EBP | 57.37c
| ||||
Modifications in this EBP are not allowed | −0.01 | −0.06 | −1.28 | −13.71 | |
Minor modifications in this EBP are allowed |
0.26
|
1.80
| 0.39 | 6.90 | |
Moderate modifications in this EPB are allowed | −0.11 | −0.67 |
0.75
|
12.92
| |
Major modifications in this EBP are allowed | −0.14 | −0.81 | 0.14 | 2.39 | |
Control over selection of EBPs | 25.62c
| ||||
Individual professionals select the EBP they will learn | −0.19 | −0.94 | 0.18 | 2.90 | |
Individual programs within agencies select the EBP they will learn |
0.18
|
1.23
|
0.44
|
8.44
| |
Individual agencies select the EBP they will learn | 0.08 | 0.44 | 0.32 | 5.97 | |
Provincial ministry mandates the EBP professionals will learn | −0.07 | −0.48 | −0.94 | −12.90 | |
Percent change to current practice | 69.71c
| ||||
Requires 25% change in current practice | −0.11 | −0.69 |
0.76
|
12.55
| |
Requires 50% change in current practice |
0.07
|
0.52
| 0.58 | 10.49 | |
Requires 75% change in current practice | 0.02 | 0.10 | −0.11 | −1.91 | |
Requires 100% change in current practice | 0.02 | 0.15 | −1.23 | −14.82 | |
Implementation process | |||||
Training focus on knowledge versus skill | 5.38 | ||||
Training focuses 100% on knowledge | −0.82 | −4.37 | −1.15 | −16.19 | |
Training focuses 67% on knowledge, 33% on step-by-step skills | 0.02 | 0.11 | 0.39 | 7.17 | |
Training focuses 33% on knowledge, 67% on step-by-step skills |
0.68
|
4.24
|
0.66
|
12.01
| |
Training focuses 100% on step-by-step skills | 0.12 | 0.85 | 0.11 | 1.95 | |
Initial training via the internet | 56.29c
| ||||
No internet learning option | −0.80 | −3.81 | 0.19 | 3.10 | |
33% of initial training can be completed online | −0.13 | −0.75 |
0.50
|
9.55
| |
66% of initial training can be completed online | 0.38 | 2.63 | 0.03 | 0.52 | |
100% of initial training can be completed online |
0.55
|
3.14
| −0.72 | −10.44 | |
Active versus passive training process | 19.18c
| ||||
Participants don’t observe, practice, nor get feedback on new skills | −0.68 | −3.37 | −1.70 | −18.10 | |
Participants observe new skills | −0.17 | −1.09 | −0.15 | −2.48 | |
Participants observe and practice new skills | 0.26 | 1.55 | 0.75 | 13.41 | |
Participants observe, practice, and get feedback on new skills |
0.60
|
3.65
|
1.11
|
18.86
| |
Follow-up training | 3.42 | ||||
Includes 0 training follow-ups | −0.69 | −3.26 | −0.82 | −12.12 | |
Includes a 1-day training follow-up |
0.38
|
2.65
| 0.21 | 3.96 | |
Includes two 1-day training follow-ups | 0.12 | 0.77 |
0.41
|
7.38
| |
Includes three 1-day training follow-ups | 0.19 | 1.26 | 0.20 | 3.66 | |
Training group size | 32.40c
| ||||
I learn this alone | −0.06 | −0.37 | −0.84 | −12.84 | |
I learn this in a group of 10 |
0.24
|
1.60
|
0.89
|
16.32
| |
I learn this in a group of 50 | 0.05 | 0.34 | 0.34 | 6.23 | |
I learn this in a group of 100 | −0.23 | −1.53 | −0.39 | −6.50 | |
Initial training time demands | 4.50 | ||||
Initial training requires 1 day | −0.23 | −1.44 | −0.03 | −0.48 | |
Initial training requires 2 days | −0.05 | −0.31 |
0.19
|
3.58
| |
Initial training requires 3 days | 0.13 | 0.89 | −0.01 | −0.26 | |
Initial training requires 4 days |
0.15
|
1.01
| −0.15 | −2.66 |
Other Measures
Experience with EBPs
Intent to Participate in Components of the Implementation Process
Procedure
Data Analysis
Results
Internal Validity
Measure | Number of latent classes | ||||
---|---|---|---|---|---|
1 | 2 | 3 | 4 | 5 | |
Parameters estimated | 42 | 85 | 128 | 171 | 214 |
Degrees of freedom | 521 | 478 | 435 | 392 | 349 |
Log-likelihood (LL) | −7741.23 | −7606.56 | −7472.18 | −7368.01 | −7285.20 |
Log-prior | −1.60 | −2.60 | −3.00 | −3.27 | −3.64 |
Log-posterior | −7742.83 | −7609.16 | −7475.19 | −7371.29 | −7288.85 |
AIC (based on LL) | 15566.46 | 15383.12 | 15200.37 | 15078.03 | 14998.41 |
AIC3 (based on LL) | 15608.46 | 15468.12 | 15328.37 | 15249.03 | 15212.41 |
BIC (based on LL) | 15748.46 | 15751.45 | 15755.03 | 15819.02 | 15925.73 |
CAIC (based on LL) | 15790.46 | 15836.45 | 15883.03 | 15990.02 | 16139.73 |
Entropy R2
| 1 | 0.692 | 0.691 | 0.693 | 0.736 |
Segment 1 (12%)
Attributes | Latent class segment | |||
---|---|---|---|---|
Segment 1 | Segment 2 | |||
R | I | R | I | |
Social context | ||||
Supervisor Support for EBP | 1 |
12.9
| 4 | 8.8 |
Colleague Support for EBP | 4 |
10.2
| 5 | 8.7 |
Trainer’s Expertise and Engagingness | 6 | 9.1 | 1 |
12.3
|
Evidence of effectiveness | ||||
Percentage of Clients Benefiting | 2 |
12.3
| 3 | 9.6 |
Effectiveness in Other Agencies | 9 | 6.9 |
6
|
7.9
|
Organizational fit of EBP | ||||
Modifiability of EBP | 11 | 2.8 | 7 |
7.4
|
Control over Selection of EBPs | 12 | 2.6 | 11 |
5.0
|
Percent Change to Existing Practice | 14 | 1.3 | 8 |
7.2
|
Implementation process | ||||
Training focus on skill versus knowledge | 3 |
10.4
| 9 | 6.6 |
Initial training via the internet | 5 |
9.4
| 13 | 4.4 |
Active versus passive training process | 7 | 9.0 | 2 |
10.2
|
Follow-up training | 8 |
7.4
| 12 | 4.4 |
Training group size | 10 | 3.2 | 10 |
6.3
|
Initial training time demand | 12 |
2.6
| 14 | 1.2 |
Segment 2 (88%)
Converging Preferences
Latent class segment | |||||
---|---|---|---|---|---|
N | % | % Segment 1 | % Segment 2 | χ2 | |
Sample size | 563 | 100 | 12 | 88 | |
Age | 3.72 | ||||
18–29 | 88 | 15.6 | 9.1 | 90.9 | |
30–39 | 188 | 33.4 | 13.3 | 86.7 | |
40–49 | 127 | 22.6 | 15.7 | 84.3 | |
50–59 | 119 | 21.1 | 9.2 | 90.8 | |
60 and older | 41 | 7.3 | 9.8 | 90.2 | |
Gender | 3.83 | ||||
Female | 455 | 80.8 | 10.8 | 89.2 | |
Male | 108 | 19.2 | 17.6 | 82.4 | |
Education | 7.65 | ||||
Graduated from college or less | 170 | 30.2 | 17.6 | 82.4 | |
Bachelor’s degree (BA or BSc) | 132 | 23.4 | 11.4 | 88.6 | |
Master’s degree | 238 | 42.3 | 8.8 | 91.2 | |
Doctoral or medical degree | 23 | 4.1 | 8.7 | 91.3 | |
Education background | 25.97* | ||||
Social work | 224 | 39.8 | 7.1 | 92.9 | |
Child and youth worker and ECE | 169 | 30.0 | 16.6 | 83.4 | |
Psychology/psychiatry/other medical training | 102 | 18.1 | 11.8 | 88.2 | |
Education | 11 | 2.0 | 27.3 | 72.7 | |
Nursing | 7 | 1.2 | 42.9 | 57.1 | |
Administration | 3 | 0.5 | 66.7 | 33.3 | |
Other | 47 | 8.3 | 8.5 | 91.5 | |
Practice setting | 1.20 | ||||
Outpatient children’s mental health service | 361 | 64.1 | 11.1 | 88.9 | |
Inpatient, residential, day treatment | 126 | 22.4 | 14.3 | 85.7 | |
Educational | 65 | 11.5 | 13.8 | 86.2 | |
Hospital | 11 | 2.0 | 9.1 | 90.9 | |
Experience | 1.59 | ||||
5 years or less | 132 | 23.4 | 12.9 | 87.1 | |
6–15 years | 204 | 36.2 | 13.7 | 86.3 | |
16–25 years | 138 | 24.5 | 10.9 | 89.1 | |
26 years or more | 89 | 15.8 | 9.0 | 91.0 |
Sensitivity analysis on support by administrators | Latent class segment | |||||
---|---|---|---|---|---|---|
Total sample | Segment 1 | Segment 2 | ||||
Approach to implementation | % | (SE) | % | (SE) | % | (SE) |
Simulation 1 | ||||||
Basic implementation | 76.6 | (0.5) | 49.2 | (0.8) | 80.3 | (0.1) |
Enhanced implementation | 23.4 | (0.5) | 50.8 | (0.8) | 19.7 | (0.1) |
Simulation 2 | ||||||
Basic implementation | 51.9 | (0.4) | 27.8 | (0.6) | 55.3 | (0.2) |
Enhanced implementation +33% supervisor support | 48.1 | (0.4) | 72.2 | (0.6) | 44.7 | (0.2) |
Simulation 3 | ||||||
Basic implementation | 35.7 | (0.2) | 23.6 | (0.3) | 37.4 | (0.1) |
Enhanced implementation +67% supervisor support | 64.3 | (0.2) | 76.4 | (0.3) | 62.7 | (0.1) |
Simulation 4 | ||||||
Basic implementation | 26.1 | (0.2) | 12.9 | (0.3) | 27.9 | (0.1) |
Enhanced implementation +100% supervisor support | 73.9 | (0.2) | 87.1 | (0.3) | 72.1 | (0.1) |