Barriers and facilitators to implementing evidence-based interventions among third sector organisations: a systematic review

A range of single studies have investigated the experiences and attitudes of TSOs in implementing EBIs and adhering to EBP [30, 31, 47‐49], investigating topics such as barriers and facilitators for TSOs to implementing EBIs [31], perceived needs of TSOs adhering to EBP [50], and the attitudes of third sector practitioners to adopting EBIs [50, 51]. Identified barriers to the implementation of EBIs by TSOs tend to involve factors related to organisational culture, such as staff resistance and organisational setting [31, 52], and factors related to the lack of resources [48]. Facilitators include having established affiliations with research institutions and employing skilled staff [47, 48]. An overarching theme of these studies seems to revolve around the notion that TSOs struggle to implement EBIs and experience serious capacity issues in becoming evidence-based providers.

To date, there has been no systematic attempt to aggregate and analyse existing research on the implementation ability of TSOs delivering EBIs. This constitutes a significant knowledge gap, given that research continues to demonstrate that implementation aspects are critical to the effectiveness of service deliveries [24, 37, 39, 53]. Also, the utility of the increasingly popular ‘evidence hubs’ and ‘blueprints’ relies on the ability of practitioners to implement the EBIs according to best practice.

The search strategy was designed to be exhaustive of the existing literature on studies investigating barriers to, and facilitators of, TSOs delivering EBIs (see Additional file 2 for search terms). A body of research addressing this topic was identified prior to the search through Google Scholar and subsequent reference checking. The bibliography of the pre-identified literature was hand-searched to identify further studies.

The following databases were searched during the systematic review using text words: ABI/INFORM Global, Applied Social Sciences Index & Abstracts (ASSIA), International Bibliography of the Social Sciences (IBSS), MEDLINE®, PAIS Index, Policy File Index, Social Services Abstracts, Worldwide Political Science Abstracts, Social Care Online, SCOPUS, and Open Grey.

The search was revised until it was sensitive enough to capture at least all the pre-identified studies. The final body of included studies was hand-searched for additional references that may not have been captured by the search. Experts in the field were contacted and websites of key organisations reviewed.

To be eligible for inclusion, studies had to be primary research or systematic reviews investigating the perspectives and/or experiences of third sector practitioners (e.g. managers, directors or service providers) with regards to the implementation/adoption of EBIs. We considered all studies that investigated the process of delivering an EBI. This process might include, but is not limited to, aspects such as fidelity to intervention protocol, and whether adaptations were made in the delivery of the intervention. We define EBIs as ‘interventions which have been tested and validated according to the principles of evidence-based practice’. EBIs may be implemented in the context of, for example, social care, healthcare, education, child services, or mental health services, but other contexts of implementation were also considered for inclusion. We define barriers as ‘any factors that obstruct the capacity for third sector organisations to implement evidence-based interventions’. We define facilitators as ‘the factors that enable the implementation of evidence-based interventions’.

All research designs were eligible for inclusion. To be eligible for inclusion, studies had to investigate factors operating as barriers and/or facilitators to the implementation of evidence-based programmes and interventions by TSOs, but this did not need to be the focus of the studies. If it was unclear whether the samples included TSOs, the authors of the studies were contacted for clarification.

Articles were screened at the title and abstract level independently by both ABM and BL using the Rayyan systematic review software [56].

To analyse the included studies, a thematic analysis was conducted, in line with best practice when aggregating data from different types of research [57‐60]. Specifically, all identified factors were identified and organised into barriers and facilitators and counted by frequency. The identified factors were then categorised following thematic analysis [60], thus enabling the synthesis to account for the arbitrary difference of factors revolving around the same underlying problem. Identified factors were only counted once per study, except for studies which identified factors specific to different subgroups (e.g. according to different organisational cases [61] or by different types of EBIs [31]). To the best of our knowledge, all included studies investigated unique samples.

The construction of themes was done inductively and entirely according to the factors identified in the data extraction. ABM conducted the full thematic analysis, which was reviewed by BL and PM on an iterative basis. All modifications were made through discussion until consensus was reached.

Additionally, we constructed two tables (Tables 5 and 6) following Rees et al. [62] to provide an overview of how the individual studies contributed to the construction of the identified themes. A study was considered to contribute to a theme if it identified at least one factor part of that theme. This enabled an assessment of whether certain studies were over- or under-represented in the thematic framework [57]. Further, it allowed for assessing the reliability of the identified themes based on an overall judgement of the quality of the studies that contributed to the individual themes. A sensitivity analysis was conducted excluding studies of low quality to test the robustness of findings.

To ensure transparency, all included studies were subject to best practice quality appraisal. For qualitative studies, a modified version of the Joanna Briggs Institute and CASP checklist was applied (Additional file 4: Appendix S3) [63]. For the appraisal of survey studies, a modified version of the AXIS checklist was employed (Additional file 4: Appendix S4). For mixed-methods studies, the appropriate quality appraisal tool was decided according to the type of method employed by the study to identify barriers and facilitators to implementation. These tools allowed for an overall assessment of key biases of the included studies, and the final quality ratings were subsequently utilised to assess the reliability of the identified factors. All appraisals were conducted independently by ABM and BL with any disagreements resolved through discussion until consensus was reached.

All included studies were published after 2009 in peer-reviewed journals, except for two doctoral dissertations [64, 65]. Of the 31 included studies, 26 employed qualitative methods [20, 30, 31, 48, 49, 51, 61, 65‐79] and 5 were cross-sectional [50, 80‐83].

In the research employing qualitative methods, 4 studies included a sample of 1–10 practitioners [61, 65, 74, 77], 16 studies included a sample of 11–50 practitioners [30, 48, 49, 52, 64, 66, 68, 69, 71‐73, 75, 76, 79, 84, 85], 2 studies included a sample of 51–75 practitioners [20, 70], and 3 studies investigated more than 100 participants [31, 51, 67] (Fig. 2). One article did not provide clear information on its sample [78]. For the quantitative articles, 1 study investigated a sample of fewer than 10 TSOs [80], 1 study included 82 TSOs [50], 1 study included a sample of 100 TSOs [50], 1 study included 112 practitioners from 41 NGOs [82], and the last study investigated 510 staff members and 296 directors [81] (Fig. 3).

The majority of the included research was conducted in the United States (US) (28/31) [20, 30, 31, 48, 49, 51, 61, 64‐67, 69‐81, 83‐85], with three studies being conducted in Chile [68], Puerto Rico [50], and Australia [82]. Most studies (23/31) referred to the included organisations as ‘community-based organisations’ (CBOs) [20, 30, 31, 48‐51, 61, 64, 65, 67, 69‐72, 74‐76, 78, 80, 81, 83, 85], with five studies using the term ‘non-profits’ [52, 66, 68, 73, 77], two referring to its included sample as ‘churches’ [79, 84] (in the US, churches are considered non-profits), and one study denoting its included organisations as NGOs [82].

All included studies investigated EBIs in the context of health and social outcomes, including addiction [31, 51, 64, 67, 83, 86], HIV [20, 30, 49, 65, 69, 71, 75, 76], exercise [66, 70, 73, 74], social work [68], cancer education [85], parenting [77], nutrition [79, 84], and with a number of studies focusing on mixed types of health EBIs [48, 50, 52, 72, 82, 85] (see Table 1). Nine studies focused on factors influencing the implementation of interventions from the diffusion of evidence-based interventions (DEBI) programme by the Center for Disease Control and Prevention (CDC) [20, 30, 49, 65, 69, 71, 72, 76, 78], and six studies investigated EBIs designed by the Center for Substance Abuse Treatment (SAMHSA/CSAT) [31, 51, 64, 67, 83, 86].

Overall, 29 of the included studies were rated to be of either high or medium quality, with only three studies being rated as low quality. Of the 26 included qualitative studies (see Additional file 5), nine were rated to be of high quality, 15 of medium quality, and two of low quality. Of the five included survey studies (see Additional file 6), four were rated to be of medium quality, and one study to be of low quality.

The synthesis identified 80 unique factors operating as barriers across 31 studies and 57 factors operating as facilitators over 24 studies. Table 2 illustrates the five most reported factors operating as barriers and facilitators to implementation of EBIs. The most reported barriers were related to recruitment and retention of service-users (14/31), problems in adapting EBIs (13/31), lack of financial resources (13/31), lack of staff resources (11/31), and implementation difficulty (9/31).

The most reported facilitating factors were related to whether the EBI matched with the mission of the TSO (9/24), flexibility for TSOs to implement the EBI (9/24), perceived effectiveness of the EBI (8/24), organisational support and prioritisation of the EBI, and supportive leadership (5/24).

The main results of the synthesis are illustrated in Tables 3 and 4, which display the prevalence of identified factors organised according to their underlying theme. All identified factors were given a reliability rating to reflect how consistently each factor was reported on and the quality of studies identifying it (see ‘Key for reliability ratings’ row).

The most prevalent themes impeding implementation revolved around resources (e.g. lack of time, finances, and staff), followed by client and community factors (e.g. recruitment/retention issues), delivery capability (e.g. lack of expertise), and organisational culture (e.g. conflict with EBI and TSO mission). Other significant themes included challenges around adapting the EBI (e.g. not knowing how to adapt the intervention to the target population) and lack of training. Less significant themes involved monitoring issues, intervention-specific problems (e.g. transportation and legal barriers), commissioning requirements (e.g. paperwork), and collaboration issues.

The thematic synthesis of factors operating as facilitators identified the main category to be organisational culture, which involved factors such as organisational support and alignment between the organisational mission and the EBI. Other significant themes included delivery capability (e.g. staff expertise), accountability (e.g. perceived effectiveness of the EBI), and adaptation (e.g. flexibility and match with target population). Less reported factors involved collaboration (e.g. with academic and experienced partners), client/community factors (e.g. community support), funders (e.g. continuous support), resources, and intervention-specific factors (e.g. availability of manuals).

To assess the consistency of the identified factors and the quality of the studies reporting them, all factors were given a reliability rating (see ‘Key for reliability rating’). Further, to ensure that the findings were not driven by the quality of the included research, we re-conducted the thematic analysis excluding the studies of low quality [77, 78, 80], which did not result in any important changes to the thematic categories or the ranking of top cited barriers and facilitators.

Tables 5 and 6 demonstrate how each study contributed to the identified themes. For both barriers and facilitators, the most reported themes were also the ones most representative of the included research. However, there were several inconsistencies between the representativeness of the themes and how often certain categories were reported on. For example, the second most reported category ‘factors related to client and community issues’ was identified in fewer studies than the third and fourth most reported categories. Further, the third most reported category ‘factor related to delivery capability’ was more representative of the included research than the second most reported category. Similarly, for facilitators the second, third, and fourth most reported categories represented roughly the same amount of studies.

There was no clear pattern to suggest that study quality or methodology affected what thematic categories to which the included studies contributed.

Additional to the analysis of factors operating as barriers and facilitators, we also aggregated the recommendations posed by the included research. The main identified themes were split between recommendations targeted at: (1) funders and collaborators, (2) research and practice, and (3) TSO practitioners. The full results can be found in Table 7, and less salient recommendation factors can be found in Additional file 4: Appendix S5.

The main recommendations for funders are to invest in technical assistance and capability training for the TSOs they fund and to assess organisational infrastructure and ability of TSOs to implement EBIs before offering funding. The main recommendations for research and practice are to have clearer guidelines on how to adapt and modify EBIs to different populations and to conduct more research on how EBIs can be modified without compromising effectiveness. The main recommendations for practitioners are to ensure capability to implement and adapt EBIs, and to invest in organisational infrastructure. These results will be addressed further in the discussion.

This systematic review identified, quality appraised, and synthesised 31 studies, most of which were conducted in the US. The thematic synthesis identified the most reported barrier for TSOs to implement EBIs to revolve around resources, in particular with respect to lack of staff and finance. The most reported category of facilitators related to organisational culture and included factors such as alignment between the mission of the TSO and the EBI, and organisational support for implementing the EBI. The findings were largely representative of the included studies and robust to study quality.

In interpreting the thematic categories, one may distinguish between capacity and capability. Capacity can be thought of as ‘how much’ an organisation can do and include aspects related to resources and infrastructure, whereas capability can be understood as ‘how well’ an organisation can do and thus relates to issues such as expertise and experience [87, 88]. Importantly, the review demonstrated that TSO practitioners experience central issues related to both capacity and capability. Specifically, the most salient barriers were related to a lack of resources, issues with clients and community (particularly in terms of recruitment and retention issues), and a lack of delivery expertise and experience.

These barriers were also reflected in the recommendations, which highlighted a need for funders to better assess TSOs before providing funding and, when having made the decision to fund an organisation, to provide continuous training and support for that organisation. For practitioners, the main recommendation was to invest in the necessary organisational infrastructure and to ensure that organisations have the technical expertise to implement an EBI. This suggests that funders often do not provide the necessary support for TSOs to implement EBIs, but also that more emphasis should be put on selecting the TSOs that are most capable and motivated to becoming evidence-based providers.

A reoccurring theme for barriers, facilitators and recommendations was that of adaptation, in which support for adaptation of EBIs was mentioned as an important facilitator and the lack of adaptation guidelines as a central barrier. Many EBIs, such as DEBIs, come in packages with clear guidelines on how they should be implemented, but do not always address the issue of adaptation and modification [89‐91]. Many studies reported that the one-size-fits-all form of EBIs often constituted a challenge, particularly regarding being unable to appropriately adapt an intervention to the target population of the TSO. This challenge is, for example, mirrored by the most reported impeding factor being ‘recruitment/retention issues’. Further, several studies reported that TSOs did not receive sufficient training and support in learning how to appropriately adapt the interventions. Some of the challenges around adaptation are reflected in the following quote:

"The question of fidelity is something that they talked about a lot… the boxed [DEBI] interventions are great, but what people really need is more technical assistance about how to effectively adapt these interventions while retaining the theoretical core. They [the agency] needs to build their capacity to understand the internal logic of the M-group piece of the intervention so that they can say ‘here is the logic of this activity, and the behavior it is seeking to address…here is our target population for this intervention…how do we change M-groups for this target population while retaining fidelity to the original design?" [20]

Past research has already emphasised the importance of theorising and outlining the ‘core components’ of interventions as part of implementation guidelines [92, 93]. However, even in clinical practice, appropriate and sufficient reporting on intervention components remains an issue [38, 94, 95]. While efforts are arguably being made to improve the reporting of interventions (e.g. through the TIDieR checklist [96]), this is a fairly recent development, and many EBIs, such as DEBIs, do not always include sufficient descriptions of intervention components [90]. Also, simply reporting on intervention components is not enough; efforts must also be made to consider whether, and if so, how, different components can be effectively adapted to different contexts and populations [97].

The challenge of EBIs sometimes being too inflexible in their form was also mirrored in the recommendations for research and practice, which proposed to better understand how one can effectively modify interventions and to provide clearer guidelines on how to adapt EBIs to different populations. Importantly, this does not mean to suggest that one should strive for standardisation of intervention components (e.g. activities), but rather that more efforts are needed to conceptualise how intervention functions or mechanisms can be tailored to different implementation settings [98].

This systematic review identified the categories reported as barriers and facilitators to be largely distinct, rather than reciprocal which is often the case in this type of review [59, 60, 99]. For example, the most reported barrier, resources, was one of the least reported factors operating as a facilitator. This finding may appear curious, considering that the included studies seem to put heavy emphasis on the lack of resources as a barrier, yet ‘more resources’ was not often identified as a facilitating factor. Similarly, by far the most reported facilitating category, organisational culture, was only the fourth most reported theme of categories operating as barriers.

One may think about this non-reciprocal relationship between barriers and facilitators through the scope of sufficient and necessary factors. That is, to improve practice, it may be necessary to provide more resources for implementation, but it may not be a sufficient step in isolation. As demonstrated by the analysis of facilitators, demonstrating internal motivation to prioritise the EBI was perceived central for successful implementation. However, regardless of the degree of organisational motivation and support, an organisation must have sufficient resources in place to become an evidence-based provider.

The idea that one must consider the implementation process in its entirety was also reflected in the list of recommendations, which entailed different suggestions for different stakeholders. Past research has consistently reported that TSOs are dependent on the requirements posed by funders and commissioners [100‐102], and that their ability to adhere to best practice is largely a function of those criteria. Thus, to improve current practice it is central to ensure that commissioning criteria are relevant and facilitating of the best possible practice.

While an increasing body of academic and policy literature contends that the third sector plays an important role in the future delivery of care and health, there seems to be limited attention to how the role of TSOs can be best facilitated. For example, little is known about whether TSOs require specific support in the delivery of EBIs compared to other types of providers. However, TSOs often report that the commissioning requirements imposed on them favour a New Public Management line of thinking, thus focussing on financial reporting and management [103, 104], which, in isolation, may overlook central aspects related to implementation and effectiveness.

To respond to the challenges identified in this review, it is important to consider perspectives from the full system of stakeholders, including practitioners, commissioners, service-users, and policymakers to discuss how TSOs can be best supported in delivering social and health services. To do so, one may draw on stakeholder inclusive methods such as consensus meetings and Delphi panels [105, 106] to determine what types of support TSOs require to appropriately deliver EBIs. Importantly, this would also allow for identifying the organisational needs and constraints of TSOs and to engage in dialogue about how the commissioning process may better assess the potential of organisations to deliver EBIs and what types of support will best enable successful implementation.

Most of the included studies were conducted in North America, and the findings of this review should be interpreted in light of that context. Similarly, the review aggregated factors from a mixed sample of organisations within health and social care and thus mainly applies to organisations of those types. Further, the review suffers from the fundamental limitation that different factors may affect the implementation of different organisations or EBIs differently. However, the studies which did investigate barriers and facilitators across different EBIs tended to identify factors that revolved around similar themes [31, 61, 76].

The synthesis was done by aggregating identified factors and thus followed a sort of ‘vote-counting’. However, by analysing and categorising the identified factors thematically, the findings should be more robust to factors revolving around the same underlying concept [107]. Also, by mapping the contribution to the categories of the included studies, the review demonstrated how representative the constructed themes were of the included research. Further, all identified studies were quality appraised, and the reliability of the identified factors were assessed according to both the consistency with which they were reported and the quality of studies from which the factors originated.

This review focused on the perspectives of practitioners following the assumption that their experiences were closest to the implementation process, but future research may consider a larger group of stakeholders, such as commissioners and policymakers. The review demonstrated that many implementation issues were grounded in the lack of adaptation expertise, driven in turn by the lack of support and expertise, and unclear/insufficient guidelines on how to adapt EBIs to different populations. To address this challenge, future research should investigate how to provide clearer guidelines for TSOs to adapt EBIs and what types of support the adaptation process requires. However, such investigations mandate careful thought on and clear outlining of what types of modifications can be made without compromising core intervention components.

This review focused on factors influencing implementation and did not discuss how these factors might correspond to other aspects, such as evaluation [108]. Becoming an evidence-based provider does not just entail implementing services developed using the best available evidence and stakeholder preferences [109], but also mandate continuous monitoring and evaluation [110, 111]. However, if an intervention is developed and tested according to best practice, is it then necessary to allocate further resources to evaluation? One might argue that requiring continuous evaluation on interventions which have already proved efficacious takes away valuable resources from service delivery. To approach this discussion, one might consider the example of mass deworming programmes, which involve medicating children against certain soil-transmitted infections in endemic areas [112]. These programmes have been—and still are—heavily implemented following an assumption of effectiveness based on single studies demonstrating promising results [113]. Yet, the utility of these interventions were recently questioned by two high-quality systematic reviews which both demonstrated null effects of the deworming intervention [114, 115]. Such findings emphasise the importance of continuous evaluation, especially for interventions that have only proved efficacious in single studies. However, to develop recommendations and guidelines on how and when implementation might influence effectiveness, more research is warranted to better theorise how implementation may moderate intervention effects and how commissioning criteria may consider such moderations.