Community capacity to acquire, assess, adapt, and apply research evidence: a survey of Ontario's HIV/AIDS sector

Community-based organizations (CBOs) are important stakeholders in the health sector [1, 2] as they not only provide a wide spectrum of programs and services to the members of their community but also play an advocacy role for broader system-level supports. As we have outlined in more detail in a previous paper [3], CBOs are typically not-for-profit organizations that: are guided by a specific mission (i.e., an overall goal) shaped by commonly held values within the community they serve; have a governance structure consisting of board members elected from the members in the community; and deliver a specific set of programs or services that are shaped by the mission and values of the organization. Because they are key health system stakeholders, it is important to support their capacity to find and use relevant and high-quality research evidence. Doing so will help ensure that programs, services, and advocacy are informed by the best available evidence.

However, there are many potential challenges related to research use. Barriers that have been consistently identified across sectors include: the complexity of research evidence, organizational barriers, lack of available time, poor access to current literature, lack of timely research, lack of experience and skills for critical appraisal, unsupportive culture for research, lack of actionable messages in research reports, and limited resources for implementation [4‐8]. Given these barriers, it is not surprising that a lack of uptake of research evidence has been noted in many different sectors [9‐13].

While there are strategies for supporting the use of research by managers of healthcare organizations and by policy makers in government [14, 15], there is still an important gap in the availability of a specific strategy for CBOs [3]. Many existing strategies for supporting the use of research evidence are based on experience and anecdotal evidence rather than on rigorous evidence of effects [8, 14, 16]. Moreover, strategies designed for supporting the use of research evidence by healthcare organizations and governments may not be relevant to the specific contexts and capacity of CBOs. The transferability of these strategies to CBOs is difficult to determine without first collecting evidence about their current capacity to find and use research evidence and then grappling with how to align capacity-building strategies with local realities [17‐21].

In order to begin to fill this gap, we conducted an assessment of the capacity of HIV/AIDS CBOs in one Canadian jurisdiction (Ontario). The role of CBOs is particularly important for addressing the HIV/AIDS epidemic in Ontario as approximately half of all people living with HIV/AIDS in Canada reside in that province [22, 23]. The province has a strong network of HIV/AIDS CBOs to respond to the local HIV epidemic, with 51 organizations serving 19 different regions at the time of this study (there are now 46 organizations) [24]. As such, this network provides essential on-the-ground support for people living with HIV/AIDS, especially for the most vulnerable [25].

We invited Executive Directors of HIV/AIDS CBOs that are funded by the AIDS Bureau of the Ontario Ministry of Health and Long-Term Care and members of the Ontario AIDS Network (n = 51) to complete an online survey. We asked each Executive Director to work with one or more managers and front-line staff to complete the survey, allowing them to respond to areas that they are most knowledgeable about but also be informed by the specific knowledge of other managers and staff.

The CBO Executive Directors in Ontario were invited to complete an adapted version of the Canadian Health Services Research Foundation's "Is Research Working for You?" survey, which has been previously validated [26, 27]. The survey is organized into four areas of assessment: (1) Acquire: Can your organization find and obtain the research findings it needs?; (2) Assess: Can your organization assess research findings to ensure they are reliable, relevant, and applicable to you?; (3) Adapt: Can your organization present the research to decision makers in a useful way?; and (4) Apply: Are there skills, structures, processes, and a culture in your organization to promote and use research findings in decision making [27]? Each domain is then broken down into subsections that ask how well their organization performs specific tasks, and each question uses a five-point Likert scale (question anchors are 1 = Don't do, 2 = Do poorly, 3 = Do inconsistently, 4 = Do with some consistency, and 5 = Do well).

We analyzed the results of the survey by calculating response frequencies for each question. We did not conduct statistical comparisons between organizations due to the small sample available for the survey. Ethics approval was obtained from the University of Toronto to complete this research.

Executive Directors from 25 organizations completed the survey (response rate = 49%). These organizations collectively provide services to approximately 32,000 clients per year with 290 full-time equivalent staff. Participant/organizational characteristics are summarized in Table 1, and the results for each question are presented in Tables 2, 3, 4 to 5. The survey respondents were predominantly from organizations with an exclusive focus on HIV/AIDS. In addition, the sample appears balanced based on region served (approximately half were based in the greater Toronto area) and includes organizations with varying scale (as measured by the number of full-time equivalent staff) and service volume (as measured by the number of clients served each year). Lastly, the organizations included in our sample provide HIV/AIDS services to a broad array of populations, including women, youth, men who have sex with men, injection drug users, specific ethno-racial communities, as well as people living with HIV/AIDS.

Overall, we found organizational capacity to acquire, assess, adapt, and apply research evidence to be low, with a couple of notable exceptions where the majority of organizations provided ratings of "Do with some consistency" or "Do well." Capacity for applying research evidence appears to be relatively strong, with approximately half of organizations indicating they "Do with some consistency" or "Do well" on all but two questions (related to time and engaging on-staff researchers) for the subdomain related to whether their decision-making processes have a place for research (Table 5, subsection VII). In addition, most CBOs indicated that they have a corporate culture that is supportive of research use (64% selected "Do with some consistency" or "Do well").

Few organizations rated any of the questions related to acquiring, assessing, or adapting research evidence higher than "Inconsistent." Areas of relative strength include identifying research evidence through peer networks, "grey" literature reports, and through websites. Capacity was lowest for the domains related to: acquiring research (subsection I); assessing the reliability, quality, relevance, and applicability of research evidence (subsections III and IV); and summarizing results in a user-friendly way (subsection V). A higher proportion of organizations selected the "Don't do" or "Do poorly" response options in these domains. Overall, the results indicate that CBOs are not only limited in skilled staff with time, incentive, and resources for summarizing results in a user-friendly way and assessing the quality and reliability of research, but they also have limited arrangements with external experts to help them.

This study gives initial insight into the capacity of CBOs to acquire, assess, adapt, and apply research in their organizations, which can provide a starting point for developing their capacity to find and use research evidence. Efforts to build capacity among CBOs should also draw on key facilitators for supporting the use of research evidence that have been cited for other groups of stakeholders, such as ongoing interactions between researchers, managers, and policy makers and ensuring research is available in a timely manner (e.g., through searchable databases of research syntheses) [7]. In addition, many strategies to support the use of research evidence highlight the importance of clear summaries that are tailored to the specific audience and highlight the take-home messages from research [7, 8, 21, 28, 29].

The primary strength of this study is that it provides a baseline assessment of CBO capacity to acquire, assess, adapt, and apply research, which can be used for identifying areas for future capacity-building strategies. In addition, the data from this study can be used in future evaluations of organizational capacity after capacity-building initiatives have been implemented. However, the results should be interpreted with caution given the low response rate and the small sample size. In addition, the generalizability of our results to other Canadian and international jurisdictions may be limited as we only surveyed organizations in one Canadian jurisdiction.

Given the limited capacity to find and use research evidence, we recommend a capacity-building strategy for HIV/AIDS CBOs that focuses on providing the tools, resources, and skills needed to help CBOs more consistently acquire, assess, adapt, and apply research evidence. Such a strategy may also be appropriate in other sectors and jurisdictions given that CBO Executive Directors in the HIV/AIDS sector in Ontario report low capacity despite being in the enviable position of having stable government infrastructure in place to support them, benefiting from long-standing investment in capacity building, and being part of an active provincial network. CBOs in other sectors and jurisdictions that have fewer supports may have comparable or lower capacity.

Capacity building does not mean that CBOs need to do everything themselves. For instance, capacity building can focus on efforts to facilitate "pull" [28], such as developing and maintaining "one-stop shopping" websites that provide access to high-quality and relevant systematic reviews of the literature with user-friendly summaries (e.g., Health Systems Evidence [http://​www.​healthsystemsevi​dence.​org] for health system managers and policy makers and http://​www.​health-evidence.​ca for public health) [30, 31], and/or "rapid response" services, such as those developed through the Ontario HIV Treatment Network for HIV/AIDS CBOs [32] and the Health Technology Inquiry Service at the Canadian Agency for Drugs and Technologies in Health [33]. Additional efforts could also include facilitating access to research evidence by providing more links to external experts who can assist CBOs with identifying relevant research evidence and assessing its quality, reliability, and local applicability. Such efforts to support the use of research evidence will require in-depth consultation with CBOs to develop approaches that reflect their specific needs.