Background
What has been done?
Objectives of the review
Methods
Search strategy
-
ABI/INFORM Global
-
Applied Social Sciences Index and Abstracts (ASSIA)
-
International Bibliography of the Social Sciences (IBSS)
-
MEDLINE®
-
PAIS Index
-
Policy File Index
-
Social Services Abstracts
-
Worldwide Political Science Abstracts
-
SCOPUS
-
Open Grey
Study selection
Screening and data extraction
-
Publication year and author
-
Study aims
-
Study design (time frame/data collection/data analysis)
-
Population (type of organisations/services/sample size)
-
Results (barriers/facilitators)
-
Other results
Data synthesis
Quality assessment
Results
Characteristics of included studies
Samples of TSOs in included studies
Size of TSOs
Purpose of TSOs
Type of evaluation
Quality of included studies
Identified barriers and facilitators
Top 3 reported barriers |
Lack of expertise and internal capability (17/24) |
Mismatch between funder requirements and appropriate goals (16/24) |
Lack of financial resources (14/24) |
Top 3 reported facilitators |
Funder requirements (6/15) |
Involvement of stakeholders to identify outcome indicators and evaluation goals (5/15) |
Training of staff and evaluation literacy (5/15) |
Weight of factors
Themes
Times cited | |
---|---|
Factors operating as barriers | |
Factors related to lack of resources | 36 |
Financial resources | 14 |
Staff resources | 6 |
Lack of time | 6 |
Lack of resources to hire external evaluators | 3 |
Resources (unspecified) | 6 |
Staff turnover | 1 |
Factors related to technical capability and evaluation skills | 36 |
Lack of expertise and internal capability | 17 |
Problems with data collection and analysis | 11 |
Inability to utilise existing data | 3 |
Difficulty conceptualising and designing evaluation | 1 |
Technical challenges | 4 |
Challenges in utilising evaluation systems and identifying outcome indicators | 27 |
Difficulty developing and using evaluation tools | 5 |
Lack of integrated systems to collect and analyse data | 9 |
Challenges in identifying accepted outcome and impact indicators | 13 |
Factors related to organisational culture and management | 26 |
No perceived benefit to conduct evaluation | 1 |
Staff resistance to evaluation | 10 |
Perceived compromise between evaluation and service delivery | 3 |
Lack of evaluation strategies and planning | 3 |
Lack of feedback between board and management staff | 1 |
Low prioritisation of evaluation | 3 |
Lack of support from board and leadership | 4 |
Evaluation not part of everyday practice | 1 |
Factors related to funder requirements | 25 |
Lack of funder requirements and support | 3 |
Differing requirements from different funders | 1 |
Mismatch between funder requirements and appropriate goals | 16 |
Poor proportioning of size of charities and funder requirements | 1 |
Micro-management by donors | 1 |
Funding insecurity (funding circles incentivising focusing on immediate outputs rather than long-term outcomes) | 3 |
Other | 5 |
Confidentiality of data | 3 |
Lack of cooperation with stakeholders | 2 |
Factors operating as facilitators | |
Factors related to receiving support to evaluate | 19 |
Partnering with evaluation experts | 3 |
Partnering with organisations working with similar activities | 1 |
Technology availability and literacy to collect and analyse data | 4 |
Benchmark data availability | 2 |
Training of staff and evaluation literacy | 5 |
Workable evaluation tools | 2 |
Having sufficient resources to evaluate | 2 |
Factors related to organisational culture and management | 18 |
Understanding internal processes | 3 |
To embed evaluation as part of everyday practice | 1 |
Improve allocation of resources | 1 |
Support from board and leadership | 4 |
Have in-house evaluation staff | 3 |
Having clear goals and evaluation strategies | 2 |
Staff support | 3 |
Positive perception of evaluation | 1 |
Factors related to the motivation to be accountable | 17 |
Involvement of stakeholders to identify outcome indicators and evaluation goals | 5 |
The motivation to influence policy | 1 |
Compare work and outcomes to others doing similar work | 1 |
The motivation to inform the sector as whole | 1 |
Improve targeting of beneficiaries | 1 |
Identify new approaches | 1 |
Ensuring control and legitimacy of activities to stakeholders | 3 |
The motivation to demonstrate and improve effectiveness of services | 4 |
Factors around funder requirements and regulations | 7 |
Funder requirements | 6 |
Regulation requirements | 1 |
Factors around economic sustainability | 5 |
Using evaluation to be eligible for funding opportunities | 3 |
Using evaluation as marketing | 2 |