Introduction
Defining a new concept for implementation—“scaling-out”
Scaling-out vs scaling-up
Key term | Definition |
---|---|
Scale-up | The deliberate effort to broaden the delivery of an EBI with the intention of reaching larger numbers of a target audience. Often an EBI scale-up will target health delivery units within the same, or very similar settings, under which the EBI has already been tested. |
Scale-out | A deliberate effort to broaden the delivery of an EBI. Scale-out is an extension of scale-up and uniquely refers to the deliberate use of strategies to implement, test, improve, and sustain an EBI as it is delivered to new populations and/or through new delivery systems that differ from those in effectiveness trials. There are three types of scale-out, each indicating the extent to which the EBI is delivered to new populations and/or through new delivery systems. |
Type I scale-out: population fixed, different delivery system | A type of scaling-out wherein an EBI is scaled-out to the same population as previously tested, but through a different delivery system. |
Type II scale-out: delivery system fixed, different population | A type of scaling-out wherein an EBI is scaled-out to a different target population through the same delivery system as previously tested. |
Type III scale-out: different population and delivery system | A type of scaling-out wherein an EBI is scaled-out to a different target population, through a different delivery system, than previously tested. |
Borrowing strength | Utilizing empirical evidence from a previous EBI effectiveness trial in combination with new evidence from a scale-out trial to test EBI effectiveness when moving it to a new population and/or through a new delivery system. Borrowing strength allows for a more limited evaluation, typically prioritizing implementation outcomes, that takes less time and expense to conduct than the original effectiveness trial. |
Intervention adaptation | Modifications to an EBI to facilitate its feasible, practical, and acceptable implementation in new contexts. |
External validity | The representativeness or generalizability of an effect. |
Core elements | Prototypical and/or necessary activities or components of an EBI. When scaling-out an EBI to a new population and/or through a new delivery system, core elements of the EBI should be retained to ensure its effectiveness. |
Empirical evidence needed for scaling-out
Two types of scaling-out: population fixed, different system, and delivery system fixed, different population, scaling-out
Type I: population fixed, different delivery system scaling-out
Type II: delivery system fixed, different population scaling-out
Evaluation options for scaling-out
EBI replicability or effectiveness
Levels of evidence for scale-out evaluations
Level Of evidence | Implementation fidelity (Implementation strategy delivered as intended) | Intervention fidelity (Clinical or health intervention delivered as intended) | Reach and exposure | Adoption | Sustainment | Effect on health outcome | Potential use |
---|---|---|---|---|---|---|---|
0: minimal or no new empirical evidence | Not measured | Training certification of facilitator and/or clinician prior to new implementation | Numbers of individuals exposed | Attendance of organizational representatives at trainings | Not measured | Not measured | Demonstration program that explicitly follows an intervention manual |
1. Proxy empirical evidence | Leadership and staff self-efficacy to support EBI | Facilitator and/or clinician ; self-assessment of fidelity | Attendance for behavioral intervention; filled prescriptions | Formal acknowledgment by organizations of adoption | Completion of yearly reports by implementing agencies | Assessment of intermediate and/or proximal health outcome | Inexpensive large-scale implementation evaluation |
2. Direct empirical evidence | Measurement of milestone attainment; speed, quality, and quantity of implementation | Independent assessment of fidelity | Ratings of quality of behavioral homework, medication adherence | Quality of staff training | Sustained number of staff and number of subjects exposed to intervention with fidelity | Change in primary health outcome from baseline | Formal implementation evaluation to establish evidence base through mediational mechanisms |
3. Full randomized hybrid trial | Evaluate intervention vs comparison on primary outcome | Type II hybrid trial to directly establish full evidence base |
Sequential mediating model for assessing EBI effectiveness
Assuming a conceptual theory of mediation holds for scaling-out
Considerations for scaling-out
Intervention adaptation
Core elements of the intervention
Discussion
Scaling-out
SYSTEM | ||||
---|---|---|---|---|
Domain (mean, regression from Fig. 1) | Same | Different | ||
POPULATION | ||||
Same |
Scaling-Up
|
Type I Scaling-Out: Population fixed, different delivery system
| ||
Implementation Fidelity (a, α) Intervention Fidelity (b, β) Reach (c, γ) Adoption (a, α) Sustainment (a, α) Health Outcome (d, e, δ) | a = 1–2, α = 0 b = 1–2, β = 0 c = 1, γ = 0 a = 1, α = 0 a = 1, α = 0 d, e = 0, δ = 0 | a = 2, α = 0 b = 1–2, β = 0 c = 1, γ = 0 a = 2, α = 0 a = 2, α = 0 d, e = 0, δ = 0 | ||
Different |
Type II Scaling-Out: Delivery System Fixed, different population
|
Type III Scaling-Out: Different Population and Delivery System
| ||
Implementation Fidelity (a, α) Intervention Fidelity (b, β) Reach (c, γ) Adoption (a, α) Sustainment (a, α) Health Outcome (d, e, δ) | a = 1,2, α = 0 b = 1–2, β = 0 c = 2, γ = 0 a = 1, α = 0 a = 1, α = 0 d, e = 0, δ = 0 | a = 2, α = 1–2 b = 2, β = 1–2 c = 2, γ = 0 a = 1, α = 0 a = 1, α = 0 d, e = 2–3, δ = 0–3 |