Plain English summary
Background
Methods
Results
Planning
School 1 Comm. 1 | School 2 Comm. 2 | School 3 Comm. 3 | School 4 Comm. 4 | School 5 Comm. 5 | School 6 Comm. 6 | Coverage and Observations while preparing the table | |
---|---|---|---|---|---|---|---|
BOLIVIA: Frequency of activities for adolescents. Target 2000 adolescents in 12 schools | |||||||
Email on SRH | 8 | 8 | 8 | 8 | 8 | 8 | Reached: 500 that had email in the 12 schools |
Bidirectional text messages | 8 | 8 | 8 | 8 | 8 | 8 | Reached: 1500 that had a phone in the 12 schools. More than 500 questions received |
Workshops in SRH | 5 | 5 | 5 | 5 | 5 | 5 | Reached: 2000. 5 topics given to all the last 4 years of school. 30 adolescents per time × 5 times. In total 475 workshops done. |
Private health consultations (Pilot study) | 9 | 9 | 9 | 9 | 9 | – | Reached: 350. Pilot for 6 months and was not implemented in the other treatment schools |
ECUADOR: Frequency of activities for adolescents. Target 2000 adolescents in 3 schools | |||||||
Sport activities | 2 | 2 | 3 | na | na | na | Reached from school to school: 50, 40 and 1310 adolescents |
Cinema on SRH | 1 | 5 | – | na | na | na | Reached from school to school: 40, 300 and 0 adolescents |
SRH Fair | 2 | 1 | 2 | na | na | na | Reached from school to school: 2600, 800 and 1600 adolescents |
Email on SRH | 5 | 5 | 5 | na | na | na | Reached from school to school: 841, 400 and 1000 adolescents |
Text messages | 1 | 1 | 1 | na | na | na | Reached from school to school: 841, 400 and 1001 adolescents |
Private health consultations | 1 | 1 | – | na | na | na | Reached from school to school: 18, 31 and 0 adolescents |
Workshops in SRH | 93 | 25 | 47 | na | na | na | Frequency is not related to the # of topics. Reached from school to school: 10,048, 20,311 and 7988 adolescents |
NICARAGUA: Frequency of activities for adolescents. Target 1500 adolescents in 6 communities | |||||||
FOY training to the community | 51 | 36 | 46 | 69 | 58 | 52 | Reached from comm. 2 comm.: 1020, 720, 920, 1038, 1160 and 1040 people |
Forum cinema on SRH | 21 | 23 | 23 | 19 | 23 | 29 | Reached from comm. 2 comm.: 315, 345, 345, 285, 345 and 435 people |
FOY training to couples | 11 | 22 | 16 | 31 | 26 | 11 | Reached from comm. 2 comm.: 22, 44, 32, 62, 52 and 22 parents, adolescents |
Workshops done by ICAS | 23 | 22 | 21 | 25 | 23 | 29 | Frequency is not related to the # of topics. Reached from comm. 2 comm.: 345, 330, 315, 375, 345 and 435 teens. |
Monitoring
“…Not all members of the project knew how to monitor activities. There was a format to check options and that is what they did but did not register what was really done, did not tell the whole process of the activity…”
“…Health promoters saw monitoring forms as something bad, as a way to control them, so they were filled as an obligation…”
1) Planning |
- No monitoring plan or no guideline to evaluate phase by phase |
- No pilot to assess feasibility of intervention package, implementers’ skills, adequacy of data |
- Interventions did not match predefined indicators |
2) Evaluation on how and to what extent the objectives are reached |
- Emphasis on administrative and financial monitoring |
- Data collected focused only on number of activities and reached people; rather than quality of service delivery or degree of behavior change |
- Single monitoring format for countries with different interventions |
- Monitoring did not include all objectives of the ToC |
- Data collected had no feedback on intervention activities |
3) Identification of failures to produce outputs |
- No monitoring to identify difficulties or flaws |
- No monitoring to describe implementation challenges |
- No report on mid-project adjustment of IP activities |
Impact evaluation
Success | Failures |
---|---|
1) Bolivia and Ecuador: Achieved modest behavioral change | 1) Data collected at t0 and t1 only for adolescents and not other stakeholders |
2) Reduced taboo, increased sensitization on SRH | 2) Inaccuracies in survey data collection |
3) Organized and improved community participation | 3) Not all data collected was used in the impact evaluation |
4) Bolivia and Ecuador: Expanded SRH networks | 4) Necessary first steps were omitted in the ToC; thus, not evaluated |
5) Tested Information and communication technology (ICT) to increase communication in SRH | 5) No multi-country intervention package but a common monitoring format that prevented good monitoring and evaluation |
6) Contributed to multi-country project design | 6) Cost-effectiveness was not assessed. Not all the activities were evaluated due to time and budget restrictions. |
Discussion and lessons learned
Summary of lessons learned |
1) Planning: theory of change
|
- Review efficient interventions, what-why-how they worked |
- Beware of time restrictions and cause-effect analysis for every activity of intervention |
- Conduct a pilot intervention to test the intervention package within the RBM framework |
- Keep a common intervention package and add activities when needed case-by case |
- Make a specific plan for community participation based on CBPR |
2) Monitoring
|
- Design guidelines for every activity to ensure effective delivery of the content |
- Monitor collection of data, phase of every activity, failures |
- Standard monitoring is useful when same intervention package across multiple sites |
- Obtain feedback on positive and negative aspects of IP and Monitoring |
- Monitor all aspects (positive/negative) of each activity |
3) Impact Evaluation
|
- Analyze success and failure factors to produce outputs/outcomes |
- Evaluate the cost-effectiveness of every intervention |
- Conduct a breakdown analysis of every intervention to understand the progress and particularities |
- Preview time and budget to analyze lessons and determine the effective interventions |