Background
Research focus
Conceptual framework
Methods
Description of the programme
Description of the macro-context
Study design: Using the steps of realist evaluation
STEP 1: Modelling the programme theory
-
district / community stakeholders: implementation of the training in schools, and support to school staff;
-
school: development of a health promoting approach, improvement of the school climate, improvement of home/school relationships;
-
staff: development of health promotion practices, promotion of partnership work, empowerment;
-
pupils: enhancement of well-being, promotion of social / physical / emotional health.
Expected outcome at district stakeholders / community level Implementation of the training in schools, and support to school staff | |
Potential factors involved | Potential mechanisms |
Institutional support
a
Involvement and support of the municipality Involvement, support and ownership of the community Engagement in community networks Stability in the team Stability in administrative structure and procedures | Importance given to HP Accession to the programme Perception of self-efficacy
b
to carry out training and support to school staff
Competency development
c
Partnership work
e
Reflexivity and sharing of experiences |
Expected outcome at school level Development of a health promoting approach, improvement of the school climate, improvement of home / school relationships | |
Potential factors involved | Potential mechanisms |
Partnership work
e
Implication of families Institutional will Support from local stakeholders Training means and resources Involvement of the community | Shared perception of HP Integration of HP in the “School Project” (School yearly policy plan
d
) Presence of a leader
Development of collective work
g
|
Expected outcome at school staff level in local schools Development of health promotion practices, promotion of partnership work, empowerment | |
Potential factors involved | Potential mechanisms |
Size of the implementation area Inclusion of training and support
Collective work
g
Institutional support
a
HP integrated in school project
d
Training means and resources Existence of HP approach within the school Perceived needs of children |
Perception of HP
h
Capacity to integrate HP in their practice
f
Accession to the programme Development of personal skills
Perceived self-efficacy
b
Capacity to use resources Motivation and interest Teachers’ empowerment
Leadership
1
Mutualisation. |
Expected outcome at children’s / pupil level Enhancement of well-being, promotion of social / physical / emotional health | |
Potential factors involved | Potential mechanisms |
School characteristics
Staff perception of HP
h
Capacity of staff to integrate HP in their practice
f
Duration and content of training (inclusion of support) Improvement of home / school relationship
Collective work
g
Development of HP whole school approach Development of HP practice Development of a supportive environment (psycho-social and physical aspects) | Implementation of health education activities Development of personal life skills
Knowledge, competency development
c
Critical thinking Involvement of children in health promotion projects |
STEP 2: Quantitative and qualitative data collection
Participants
Instrumentation
-
semi-structured interviews with district pedagogical advisors from two educational districts, three programme coordinators, and a selection of teachers in four schools;
-
written documents relating to the programme: minutes of steering meetings (3), minutes of operational team meetings (12), programme presentations (2–6 months apart), intermediate evaluation reports (1), training session evaluation reports (2), results from state of play questionnaire targeting teachers. Documents relating to school actions and practices were also collected. Such documents included minutes of school board (seven in school A, six in school C), school project action sheets (four in school B), teaching session preparation sheet (one in school A), and written documents as well as pictures by pupils.
Data analysis
STEP 3: Context-mechanism-outcome (CMO) configurations
Results
CMO configurations at the three levels of implementation
Level A: CMO configurations relating to outcomes at educational district and community stakeholder level
Key factor: Implication of trainers at district and community level | |
Moderator (mod) | Effect on implication |
Initiation by decision-maker | Positive effect: facilitated coordination and upscaled available resources. End result was enhanced implication. Negative effect: reluctance of staff to participate in the programme was due to a concern that their practice would be instrumentalized by political decision-makers. Participation was sometimes perceived as being imposed. |
Level of delegation from institutional hierarchy | Positive effect: more room for initiative on the part of district teams led to higher implication. The level of delegation was related to the intention of protecting the teams from an “intrusive” programme. |
Institutional support from Head of Regional Education Authority | Positive effect: promoted active participation of the health and social district departments. Negative effect: implementation was perceived as following a top-down mode of introduction, and adding constraints. This created reluctance to participate in the programme. Counter-balance: development of relationships between participants counterbalanced the fact that the programme was in most cases imposed by staff at higher decision-making levels. This factors enhanced motivation and implication. |
Implementation area | Negative effect: the area referred to electoral districts and not national education districts. This led to tensions and reluctance as all the schools which related to the same school district could not be enrolled. |
Name of programme | Negative effect: the name was unclear and determined reluctance to participate in the programme on the part of district teams |
Key factor: Appreciation and accession to the programme | |
Moderator | Effect on appreciation |
Programme content | Positive effect: overall, the programme was appreciated due to concreteness of its content “for the first time, we went into pragmatic and concrete things”. Negative effect: suggestions for programme development include: more focus on pragmatic issues, more focus on the needs of the different schools at the very start of the programme. Only one district focused on how staff could engage in the use of the tools introduced during the training. The development of specific content depended on the district staff’s competency (e.g. relaxation sessions). This situation affected school staff’s appreciation of the programme negatively. |
Training | Positive effect: inclusion of training in the programme enhanced accession to the programme as staff felt the programme was in line with their expectations. |
Key factor: Development of partnerships | |
Moderator | Effect on development of partnerships |
Institutional support | Positive effect: support from the head of the Regional Education Authority was beneficial to the implementation of the training, and the development of partnership work with the Teacher Training College. |
Implication of the Teacher Training College. | Positive effect: implication of the Teacher Training College in train-the trainer sessions had a positive effect on the initiation of partnerships. Moderating factor to implication of the training college: low implication of the Teacher Training Colleges was due to organizational and communication issues between the two institutions. |
Level B CMO configurations relating to the outcomes at school level
Level C CMO configurations relating to outcomes at school staff level
-
Outcome 1: Participation of school staff in the training
Outcome 1: Participation of staff to training | |
---|---|
Key contextual factor | Effect on participation |
Needs assessment and adaptation of the programme | Positive effect: oriented the choice of approach used in the training (e.g. cognitive perspective in line with teachers’ needs) and determined changes and evolution of the training Positive effect 2: the adaptation of the programme to the needs of school staff had a counter-balance effect on the reluctance to participate in the programme, in cases where school staff perceived their participation as being imposed by institutional decision-makers |
Importance given by district teams to the programme | Proportional effect: higher priority given to the programme by district staff determined higher participation of school staff. Support from the district management team enabled the training to actually take place in some schools. |
Availability of staff | Positive effect: when staff were available they could participate. Negative effect: the means to replace school staff who were participating in the training were not provided. The programme was perceived as an add-on activity in school staff’s busy schedule |
Means allocated | Negative effect: insufficient means were allocated which hindered the implementation of the training |
Implication of school staff | Positive effect: higher implication of school staff was linked with higher participation of school staff in the training |
-
Outcome 2: Change in school staff’s health promotion practices.
CMO configurations in the selection of four schools
School A: Small rural school (three classes), very high socio-economic status
School | Contextual factors influencing the implementation process | Outcome | |||||
---|---|---|---|---|---|---|---|
Leadership of school principal | Team cohesion and collective work | Availability of staff | Home school relationship | Institutional support | Motivation | ||
A Small rural school Very high SES |
++
|
++
|
-
Key factor
|
++
|
N/A
|
N/A
| Seldom use of tools Inclusion of action in school project Few partnerships |
B Large urban school Medium SES |
++
|
-
Key factor
|
-
Key factor
|
++
|
++
|
-
Key factor
| Few members trained Difficult implementation Programme perceived as add-on and time-consuming Very limited impact |
C Large urban school Medium SES Social diversity |
–
|
+/−
|
N/A
|
-
Key factor | – | N/A | Programme not considered a priority No assessment of programme impact |
D Medium suburban school Low SES | +/− | + | N/A | - Key factor | +/− | – | Moderate impact and difficult implementation New partnerships |
SCHOOL B: Large urban school (ten classes), medium socio-economic status
SCHOOL C: Large urban school (14 classes), medium socio-economic status, social diversity
SCHOOL D: Medium suburban school (six classes), low socio-economic status
STEP 4: Feedback on initial theory
Outcomes, mechanisms and contextual factors
Combinations and interactions between factors
-
whether the programme generates expected outcomes or outputs at the different levels of implementation;
-
how the programme operates at the different levels of implementation;
-
how other factors influence the process, i.e. one factor may moderate, potentiate or counter-balance the effect of another factor on the process;
-
how the programme evolves and is sustained, as some factors retroact on the programme.