Background
NPT construct | Definition | Sub constructs |
---|---|---|
Coherence | The sense-making work that people do individually and collectively when they are faced with the problem of operationalizing a set of practices. | • Differentiation • Communal specification • Individual specification • Internalisation |
Cognitive Participation | The relational work that people do to build and sustain a community of practice around a new technology or a complex intervention. | • Initiation • Enrolment • Legitimation • Activation |
Collective Action | The operational work that people do to enact a set of practices, whether these represent a new technology or a complex healthcare intervention. | • Interactional Workability • Relational Integration • Skill set Workability • Contextual Integration |
Reflexive Monitoring | The appraisal work that people do to assess and understand the ways that a new set of practices affect them and the others around them. | • Systematisation • Communal appraisal • Individual appraisal • Reconfiguration |
Aim
Methods
Study inclusion and exclusion criteria
Literature searches
Databases | Search terms—combined with ‘AND’
| |||
---|---|---|---|---|
School | Implementation | Intervention/change | Health | |
Cochrane Library EMBASE Via OVID ERIC Via EBSCO host Medline Via EBSCO host SCOPUS Web of Science Via Thomson Reuters | school* | implement* OR adopt* OR integrate* OR normali* | improvement* OR innovation OR knowledge* OR organisational change* OR quality improvement OR readiness to change* OR behaviour change* OR intervention* OR school based intervention* | health* |
CINAHL Via EBSCO host | school* | implement* OR adopt* OR integrate* OR normali* | CINAHL Search Terms: Health Behaviour exp. OR Behavioural Changes OR Behaviour Modification exp. OR Health Education Key Words: improvement* OR innovation OR knowledge* OR organisational change* OR quality improvement OR readiness to change* OR behaviour change* OR intervention* OR school based intervention* | health* |
PSYCHINFO Via EBSCO host | school* | implement* OR adopt* OR integrate* OR normali* | MeSH Terms: Behaviour Change OR Health Education OR School Based Intervention Key Words: improvement* OR innovation* OR knowledge* OR organisational change* OR quality improvement OR readiness to change* OR behaviour change* OR intervention* OR school based intervention* | health* |
Study selection
Data extraction
Data synthesis
Quality assessment
Results
Study | Intervention | Study design | Population | Implementation measurement | Data analysis | Key results—factors affecting implementation
| Quality appraisal |
---|---|---|---|---|---|---|---|
Audrey et al., 2008 [34] UK |
Tobacco
A Stop Smoking In Schools Trial (ASSIST) | cRCT. questionnaire interviews | 30 ASSIST schools & 29 control | Process evaluation to examine the context, implementation and receipt of the intervention | Framework method of data management. (reading, coding & identifying themes, & sorting material according to key issues) | Teachers welcomed external training—it interested pupils, prevented difficulties of discussing smoking with teachers and relieved staff burden. Implementation appeared compatible with the school ethos and timetable. Smoking was perceived as a difficult issue and staff welcomed a new initiative. Disruption to the timetable was inevitable, and the importance of communication between ASSIST staff and teachers was important | CASP: Moderate |
Barr et al., 2002 [33] USA |
Tobacco
Tobacco Use Prevention Education (TUPE) | Telephone survey | 296 middle school teachers & 282 high school teachers | Relations between TUPE teachers’ receptivity or amenability to implement TUPE programs and features of implementation settings | Cluster analyses for amenability to implementation. A one-way ANOVA for associations between amenability and implementation. A hierarchal multiple-regression for staff effectiveness perceptions | Indicators of staff amenability were variable. The most amenable staff reported consistently covering each activity with few barriers. For staff perceptions of effectiveness to prevent smoking initiation: Tobacco related norms accounted for 9.9% of variance, staff training & TUPE support or barriers—4.2%, and class activities—4.0%. For staff perceptions of TUPE for cessation: Tobacco norms—6.6% of variance, staff training & TUPE support—6.3%, class activities—5.5% | EPHPP: Moderate |
Basen-Engquist et al., 1994 [31] USA |
Tobacco.
Minnesota Smoking Prevention (MSPP) | Questionnaire | 39 districts in live training & 33 in video training. Mean number of pupils was 41, 2.8 teachers | Assessing how the type of teacher training affects implementation via a live workshop or video training | Fisher exact test & Mann-Whitney U for differences in teacher implementation Two group t-tests tested differences between students in the live and video districts | The relationship between type of training and use of the curriculum was significant. Districts who were assigned to the video training condition were less likely to teach the curriculum. However, implementing teachers from both groups reported high levels of implementation. Students in live workshops were more likely to recall discussions and activities | EPHPP: Weak |
Garrahan 1995 [40] USA |
Substance Use
| Systems approach model | 800 students | Not stated | Baseline substance use data was collected via a survey & analysed | Involving school personnel in a building-wide manner and monitoring efforts and outcomes was important. All implemented intervention aspects were linked to existing components of the school, and this gave the impression that what was implemented was based on common sense or self-evident reasoning | EPHPP: Weak |
Jarrett et al., 2009 [35] USA |
Tobacco
Not-On-Tobacco (NOT) | Survey | 769 pupils who reported regular smoking. | Perceptions of facilitator characteristics & the relationship between perceptions & outcomes | Descriptive analyses used to determine overall ranking of facilitator characteristics. Chi-square test to determine if facilitator ratings differed by race or sex | 88.7% of pupils rated facilitators as favourable. No nagging or preaching, nonjudgmental, trustworthy, caring, & confidentiality were scored highly. There were few differences in ratings by race. Favorability scores were associated with changes in smoking (quit or reduce). Pupils who perceived facilitators favourably showed significant smoking reduction and cessation rates, regardless of sex or race | EPHPP: Weak |
MacDonald and Green 2001 [44] Canada |
Substance Misuse
| Interviews and observations with Project Workers (PWs) | 100 interviews in 6 sites with school admins, teachers, pupils, parents, & agency staff | Participants were probed around the level of understanding and support for prevention, implementation experiences, implementation barriers & facilitators, support for PWs and the school’s problem with drug and alcohol issues | Constant comparative method of grounded theory Field notes were recorded and used to support analysis | PWs needed to establish legitimacy and familiarity within schools, by overcoming staff opposition. They had to address conflicting expectations, resulting from poor preparation. Schools had to be ready and willing to implement, and PWs faced issues selling the model, and facilitating participation. Training sought to teach PWs to understand the model, but this did not occur and PWs realised they did not understand it enough to implement to others and few achieved it as intended. Some tried, but were discouraged by school barriers. Some retained key features, but omitted elements due to admin pressure or context demands | CASP: Strong |
McBride et al., 2002 [43] Australia |
Alcohol
The School Health & Alcohol Harm Reduction Project (SHAHRP) | Longitudinal study | 41 classes.28 teachers 6 schools | Series of methods to optimise and assess implementation fidelity including training, critical assessment and self report | Spearman’s rank measured fidelityTheme matrices described qualitative responses | SHAHRP was taught 80.7% as intended, with fidelity ranging from 78.9 to 83.4%. Implementation was optimised by: training, staff and pupil motivation and timing. Teachers found too much work in some lessons, interruptions reduced classroom time and implementation effectiveness was pupil dependent. Expectations needed to be lowered for difficult pupils and some activities were not implemented as intended | CASP: Weak EPHPP: Strong/Moderate |
McCormick et al., 1995 [32] USA |
Tobacco
| RCT | 21 districts, 50 schools, and 3000 pupils. Districts were assigned to control or intervention. | Use of ‘Level of use’ tool and implementation check-sheets | Population means, median, frequencies & correlations used for summary. Non-parametric tests tested for differences between control and intervention | Overall implementation completeness was low, with the mean % implemented being 70% and 23% implemented ≥90%. Larger districts were more likely to implement than small ones. Districts with favourable climates were more likely to implement and reported higher usage. Trained teachers were more likely to implement curricula and more likely to implement higher proportion | EPHPP: Moderate |
Pettigrew et al., 2013 [42] USA |
Substance use
keepin’ it Real (kiR) | Ethnography | 39 schools; 14 Control, 14 Rural: Mean number of pupils per school = 99, with a range from 27 to 226 | An assessment of teacher implementation using the indicators; delivery methods, consistency of delivery, teaching standards | Coding provided; quantitative implementation ratings—quality adherence, adaptation, delivery and engagement, whilst qualitative codes identified adaptation and engagement | Analysis identified teacher control as passive, coordinated, or strict, and pupil participation as disconnected, attentive, or participatory; serving as a classroom typology for kiR implementation. Passive teachers were linked with passive pupils, strict teachers had attentive pupils, whilst classes with participatory pupils were taught by coordinated teachers. Teachers who taught kiR frequently tended to display similar control and pupils participated consistently | CASP: Moderate |
Rohrbach et al., 2007 [37] USA |
Substance Use
Project Towards No Drug Abuse (TND) | RCT | 18 schools—6 in each different condition. Pupils ranged from 13 to 19 years of age | Study compared teachers with Program Specialists (PSs). Questionnaire assessed implementation fidelity of TND via adherence, classroom process and perceived pupil acceptance | Inter-rater reliability was calculated for each item. To test the effect of implementer on fidelity and outcomes, a mixed-linear model was used | Of the 4 indexes of fidelity, only delivery quality differed between PSs and teachers. Both teachers and PSs achieved effects on 3 of the 5 immediate outcome measures, including program knowledge, addiction concern, and self-control. Pupils’ posttest ratings of the program and the quality of delivery showed no difference between teacher and specialist-led classrooms | EPHPP: Moderate |
Skara et al., 2005 [36] USA |
Substance Use
Project Towards No Drug Abuse (TND). | Questionnaire | 18 schools—6 in each different condition. 2735 students completed pretest questionnaires: 85% completed post-program | Questionnaire assessed implementation fidelity of TND via questions open and closed questions | Data was analysed using a generalised mixed-linear model using SAS | The curriculum was implemented as intended, received favourable ratings, and significantly improved knowledge. Providers reported high adherence to lesson plans and lessons were not difficult to teach. Adherence and delivery quality did not differ by curriculum or school. Individual ratings of delivery quality were favourable, including providers’ perceptions of pupil participation, pupil interest, provider’s maintenance of class control & providers’ perceptions of effectiveness | EPHPP: Moderate |
Sloboda et al., 2009 [39] USA |
Substance Use
Take Charge of Your Life (TCYL) | Observation and surveys | TCYL was delivered by 140 Drug Abuse Resistance Education (DARE) officer instructors | Implementation fidelity measured using instructional strategy (IS) | Descriptive statistics & analyses between content coverage and IS & scores from targeted lessons were conducted using hierarchical linear modelling to gain 2-level random intercept models | Higher content was correlated with IS. There were no correlation between age, sex, race, education, content coverage or use of IS. Pupils with higher coverage scored higher on the consequences measure. Results indicated pupils with a higher proportion of the content had greater perceptions of negative consequences. Greater exposure and greater content coverage was related to negative alcohol expectancies | EPHPP: Weak |
Stead et al., 2007 [38] UK |
Substance Use
Blueprint | Observations and interviews | 30 schools in 4 Local Authority areas: 24 intervention & 6 control. Year 7 (11–12 years) & Year 8 (12–13 years) | Implementation fidelity measured via adherence, exposure, participant responsiveness, quality of delivery and program differentiation | Observation schedule used to generate descriptive statistics. | The mean content fidelity was 72%. As teachers got familiar with lessons, they were likely to modify or omit elements. Fidelity was highest in teacher-pupil lessons & lowest for pupil-pupil. Resource use was variable and teachers found timing and completing content difficult. Teachers were unsure of interactive sessions due to disruption & unpredictable outcomes. Some teachers expressed concern about answering questions about drugs, but there was no difference in delivery quality of teachers with experience & those without | CASP: Moderate |
Sussman et al., 1993 [30] USA |
Tobacco
Project Towards No Tobacco Use | Questionnaire | 4852 7th grade pupils. 9 Health Educators. 76 observers collected teacher data | Key implementation measures were around program completion and delivery (fidelity- adherence, exposure, reinvention) | Pupils & educators gave ratings of implementation. Post hoc comparisons were used between pairs of means and one-way ANOVAs predicted response means | Adherence did not vary by condition and high levels of implementation were observed in all conditions. Pupils preferred physical consequences and enthusiasm was rated the lowest. Health educators’ enthusiasm, effort and class enthusiasm differed by condition. Teachers did not differ in their ratings of class control or understandability | EPHPP: Weak |
Thaker et al. 2008 [41] USA |
Substance Use
Reconnecting Youth (RY) program | Organisational diffusion study | At risk of drop out students from grades 9–12. 5 schools from each district took part | Three diffusion of innovation indicators used: perceived advantage, complexity and compatibility. Capacity, school turbulence and leadership/admin support were also explored to assess how they could affect implementation | Survey data was analysed using SPSS whereas interview data was transcribed and analysed using qualitative content analysis | Teachers reported learning RY difficult, as they were not prepared & needed to plan. RY was rigid, complex and difficult to implement the timelines & content. School capacity (skills and resources) varied & affected implementation. Other issues were budget shortfalls, funding cuts, difficulties finding rooms and school turbulence (transient pupil populations, school reorganisation, schedule changes, & staff turnover). RY lacked leadership and admin support. Only 50% of staff reported principles being supportive. Whilst only 1/3 of district admins considered RY important | EPHPP: Weak |
Results of quality assessment
Synthesis of results
Factors affecting implementation | Papers | NPT construct |
---|---|---|
Distinguishing from current practice | Coherence | |
Fitting with school ethos | [34] | Coherence |
Providers seeing the value or benefit of an intervention | Coherence | |
Providers not delivering or not understanding how to deliver (use of specialist knowledge) | Coherence Collective Action | |
Training | Coherence Collective Action | |
Implementation driving force | Cognitive Participation | |
Role identity—provider ‘agreeing’ it should be part of their role | Cognitive Participation | |
Provider supporting intervention | Cognitive Participation | |
Provider motivation | [43] | Cognitive Participation |
Sustainability | [30] | Cognitive Participation |
Young people behaviour | [42] | Cognitive Participation |
Providers feeling uncomfortable with delivery | [38] | Cognitive Participation Collective Action |
Budget cuts or limited resources | [41] | Collective Action |
Disruption to school timetable | [34] | Collective Action |
Favourable organisational climate/host support | Collective Action | |
Fidelity | Collective Action | |
Importance of staff skills, knowledge or characteristics | Collective Action | |
Involving schools; monitoring outcomes | [40] | Collective Action |
Schools prepared for implementation | [44] | Collective Action |
Staff turnover | [41] | Collective Action |
Modifying practice (from feedback) | [38] | Reflexive monitoring |
Negative implementation experience | [41] | Reflexive monitoring |
Positive feedback | [36] | Reflexive monitoring |