I-RREACH: an engagement and assessment tool for improving implementation readiness of researchers, organizations and communities in complex interventions

The availability of clinical practice guidelines for cardiovascular disease is necessary but not sufficient alone to bring about significant improvements in cardiovascular health. Risk factor reduction is complex and requires lifestyle modifications, behaviour changes and pharmacologic therapy. Health care policies may actually contribute additional barriers to evidence-based care and changes to health systems, and these policies may be required to improve the prevention and management of chronic diseases [8].

Facing the mounting burden of CVD and related complications, LMICs and Aboriginal populations often lack the financial means and/or geographically accessible services for diagnosis, treatment and management options, leading to increased rates of morbidity and mortality [1]. In Canada for example, studies show that death rates from circulatory system diseases among Aboriginal people are significantly elevated compared with non-Aboriginal people [9]. Hypertension, one of the major risk factors for CVD, is highly prevalent and has no noticeable symptoms; therefore, it is too often undiagnosed and goes untreated in low-resource and low access health care environments where the focus is on other more urgent or competing medical problems. A recent systematic review in Canadian Aboriginal communities found an average prevalence for hypertension of 23.5% [10]. High rates of undiagnosed hypertension [11] were also linked to elevated rates of chronic kidney disease [12]. Barriers to care include lower access to renal dialysis and transplantation [13].

Although geographically and culturally quite dissimilar to Canadian Aboriginal populations, resource-poor African countries such as Tanzania struggle with similar predicaments. While there are no national data sets on the rates of hypertension, current evidence points to higher hypertension prevalence with dramatically lower treatment and control rates [14] associated with higher stroke rates [15]. Compared to rural and remote Canadian communities, the lower access to hypertension management in Tanzania is due, in large part, to financial resource constraints limiting numbers of adequately trained health care providers, diagnosis and treatment as well as a lack of infrastructure connecting patients, their medical records and their health care providers [16].

DREAM-GLOBAL (Diagnosing hypeRtension-Engaging Action and Management in Getting LOwer Bp in Aboriginal and LMIC) is a research project designed to increase our knowledge of how to develop and implement affordable, evidence-based, guidelines-driven hypertension management interventions at the patient, provider and community level in LMICs (Tanzania) and Aboriginal communities in Canada by leveraging innovative technologies, health services and research methodology. The project is informed by the Canadian Hypertension Education Program (CHEP) clinical practice guidelines drawing on evidence for high blood pressure prevention through dietary sodium restriction, blood pressure measurement, education interventions for health care providers and patients, inter-professional care, health systems and community-based interventions including automated reminder systems [17,18]. It combines a multilayered approach including many elements of the chronic care management model, as well as task shifting from health care providers (HCP) to community health resource and community health workers. The project also facilitates self-management support, decision support, delivery system design, clinical information systems, while building on existing capacity in health care organizations and community resources [19]. DREAM–GLOBAL uses a randomized controlled trial (RCT) [20] design to test the effectiveness of a mobile health strategy using cell phone-based short message service (SMS) technologies to facilitate blood pressure measurements and feedback between patient and provider through the local existing mobile phone network. The flow diagram in Figure 1 displays the planned progression of participants through the trial. The study is a complex intervention leading to changes in the behaviour of patients, providers and local systems. It is implemented using a collaborative community-based participatory approach.

The components of the DREAM-GLOBAL intervention are evidence-based, but we recognize that interventions can easily fall short of achieving the intended outcomes due to ineffective implementation and delivery in their ‘real life’ application within the health care system. In particular, low-resource environments are strongly affected by implementation issues. This phenomenon, known as the implementation gap, is linked to the lack of understanding of the multifarious factors that may impact on the effectiveness of an intervention when implemented at the community or population level [21]. Confounding factors include the uniqueness of local power structures, health systems, community organizations and human actors making standardization unrealistic.

There is an increasing acknowledgement that to measure the degree of effectiveness of an intervention in real clinical practice, an RCT must be pragmatic, maximizing external validity (generalizability) while trying to preserve internal validity (reliability and accuracy) [22]. A traditional RCT design does not offer an analysis of the multiple factors that impact on outcomes nor how or why the observed impact occurred [23]. Leykum and colleagues argue that this leads to a profound dilemma: “How do we design interventional trials that are generalizable, but also have enough flexibility to be meaningful and more likely to be successful locally?” [24] Research leading to a better understanding of key implementation factors and the mechanism by which these influence outcomes are needed to guide implementation strategies for intervention trials as well as proven interventions in diverse environments.

Given the rising rates of CVD in disadvantaged populations, it is essential to study the implementation factors of CVD interventions in low-resource environments where they are urgently needed rather than extrapolating from high-income settings [21] The DREAM-GLOBAL intervention trial is planned for diverse communities in Tanzania and Aboriginal communities in Canada, and we therefore explored these key implementation issues through participatory research prior to rolling out the study. Our objective was to use the information gathered to develop an implementation tool to guide the process of identifying key implementation issues related to people, community, health systems, culture and relationships that are likely to differ between settings. As each of these factors introduces complexity, our goal was to design the implementation tool to be easily adaptable to various health care environments and health issues in a variety of communities globally.

Using community-based participatory research (CBPR), we aimed to develop a tool to guide dialogue between researchers and communities on key aspects of researcher, community, system functionality and readiness. We report on (1) the theoretical underpinnings of this tool, (2) the key domains of complexity and issues and how these are uncovered by this tool, and (3) the evaluation feedback from stakeholders during the testing of the tool.

Our methodology for the development of the tool consisted of a three-pronged CBPR approach: We employed a consensus method to (1) develop the theoretical basis of the tool and (2) to identify key implementation factor domains; then, (3) we evaluated the tool.

CBPR is not a research methodology but rather a value-based approach to inquiry. It has been defined as “a collaborative research approach that is designed to ensure and establish structures for participation by communities affected by the issue being studied, representatives of organizations, and researchers in all aspects of the research process to improve health and well-being through taking action, including social change” [25]. Israel and colleagues articulate several key principles that define CBPR. These principles include notions such as recognizing the community as a unit of identity, building on community strengths, collaboration in all phases of the research, integration of research and action based on co-learning and knowledge sharing to address inequalities [26].

Translational and intervention research in low-resource environments frequently has a distinct cross-cultural aspect, with much of the scientific evidence provided by cultural outsiders. Rivkin and co-workers argue that this can pose challenges for intervention science as health concepts used in scientific research may not translate or have different meanings in culturally diverse partner communities [27]. Trickett and others have further elaborated that “culture pervades all aspects of community interventions” and community collaboration are fundamental to the success of complex interventions [28]. Warry noted that collaborative research in these settings requires researchers to relinquish control over the research process in order to strengthen their explanations [29].

In this study, the CBPR approach formally began when community stakeholders interested in exploring the implementation of DREAM-GLOBAL were invited to share their perspective on the development of an implementation tool based on their lived experience within partner communities using a consensus method.

A consensus method informed by Indigenous values [30] and CBPR principles was used to initiate a dialogue involving academic researchers and community stakeholders addressing potential implementation issues for DREAM-GLOBAL. Community stakeholders consisted of people with lived experience and close connections in the targeted partner communities. They included health care workers and administrators, community-based researchers, health champions and those who were or had family affected by hypertension.

The Indigenous consensus method that we employed differs from traditional academic consensus methods such as the Delphi method [31] as it does not privilege the perspective of academic experts. Instead, the process invites cultural values and knowledge gained from the lived experience of participants, by eliciting the perspectives of those who are eventually affected by the intervention such as community members and local communities of practice. The consensus process is completed once the nexus between community, academic and practice-based knowledge manifests, which in this case was achieved after three consensus cycles (see Table 1).

The finalized I-RREACH (Intervention and Research Readiness Engagement and Assessment of Community Health Care) tool is an implementation tool that includes a community fact sheet, an interview, and focus group guide designed to elicit information on key factors that may impact the implementation of the intervention. It was implemented during visits to participating communities in Canada and Tanzania. At the end of each interview and focus group, participants were asked to complete a short anonymous evaluation form rating their immediate experience with the I-RREACH process. The evaluation tool was designed to gather information on a five-point scale on how participants perceived the following: (1) the clarity of questions, (2) rapport and understanding between researcher and themselves, (3) improvements in understanding of the intervention and implementation issues, (4) cultural safety, (5) I-RREACH process as a facilitation tool for knowledge exchange for implementation of the intervention and (6) any comments or additional suggestions for improvements. The evaluation form is provided in Table 2. It is scaled to indicate high levels of satisfaction when participants either “agree” or “strongly agree” with the provided statements. It can be easily adapted to I-RREACH evaluations by researchers who wish to use this tool in the future. To summarize the data, we tallied up scores for each of the five scored domains and thematically summarized the short responses and suggestions for improvements. While any dissatisfaction is reason for concern that researchers may need to explore, we deemed the I-RREACH tool to be acceptable from the participants’ perspective if at least 85% of scores were in the combined “agree” and “strongly agree” categories and no major concerns were raised in the open-ended question component of the questionnaire.

In line with the community-based participatory nature of the development of the I-RREACH tool, the study protocol was submitted for academic ethics review and community-based REB review (in Canada). Members of the research team also sought formal approval by decision makers in each of the participating communities after local, in-person presentations. Academic and governmental ethics approvals include the following: The National Institute for Medical Research in Dar es Salaam, Tanzania (approved March 19, 2014); Queen’s University Health Sciences and Affiliated Teaching Hospitals Research Ethics Board, Kingston, Ontario (DMED-1603-13 approved June 21, 2013); and Sunnybrook Health Sciences Centre Research Ethics Board, Toronto, Ontario, (#182-2013, approved May 31, 2013). Community-based ethics review in Aboriginal communities included The Cree Board of Health and Social Services of James Bay, Ontario (approved September 11, 2013) and Manitoulin Anishinaabek Research Review Committee (MARRC) Ontario (approved October 7, 2013). The study was also formally approved by decision making bodies in all participating communities through Village Councils approvals in Tanzania and Band Councils Resolutions in First Nations in Canada.

The information gathering and dialogue tool we developed consists of three stages: (1) a community profile section with a fact table which can be populated through various fact-finding activities, (2) an interview guide to help facilitate the discussion and understanding of strategic topics with key stakeholders in the community and (3) a focus group guide to lead a dialogue on community-centered issues.

This section of the I-RREACH tool is designed to be employed at the start of the engagement phase. This section gathers relevant community profile information to determine if a specific community has the basic characteristics that would enable implementation of the intervention. Information gathering is focused on the community, relevant governance, health care staff, structure and programs, infrastructure and technology as well as resources. This data may be found in publically available records and during initial contacts with the community. When this part of the I-RREACH tool is used, it may not be clear yet if the implementation will occur, either because the community has not yet fully affirmed interest or because certain characteristics identified in the community profile may exclude the community from participation. For example, in the DREAM-GLOBAL study, cell phones are used to send educational SMS text messages to patients and lack of cell phone coverage would therefore disqualify a community from participation. Other studies may require specific health care capacities or community demographics, all of which would be identified during this early data collection phase.

During this phase of I-RREACH, a dialogue with key stakeholders is established. We learned that key stakeholders should be identified early on in collaboration with the local contacts. Key stakeholders generally are individuals who could have a negative or positive effect on the implementation of the intervention. Many of these key stakeholders may be affected by the intervention through training, changes in practice or additional responsibilities. Others may be decision makers, health champions or people personally affected by the health issue or they may wield influence in the community. The discussion sessions with key stakeholders in this phase builds on the information gathered in the community profile but elicits the more practical and community-specific knowledge that is needed for effective implementation from an operational perspective. The interview phase also allows stakeholders to learn about the intervention and reflect on their expectations and level comfort with the project. The interview domains cover characteristics of local leadership, descriptions of services, access and awareness of relevant community programs, local understanding of the health issues, resources and planning, the perceived fit between the intervention and community goals and finally the requirements and preparedness of researchers and community to conduct collaborative research and implementation of the intervention. Interviewers should continually probe for information that would lead them to a higher level of understanding of the role of politics, policy and resource allocation on the health issue in question; the cultural context and community life; local interaction of medical traditions; oppressive hegemonic experiences in the community with health services or research; and community expectations related to the research and the academic team.

The focus group discussions with community members cover similar issues as the topics discussed in the previous phase; however, the guide is designed to elicit the lived experience of those who are expected to be the main recipients of the intervention. The discussion framework explores the qualities of community life and history and how the health issue is culturally situated, such as the intertwining of health and social issues as well as expectations of community members. Providing a safe environment to exchange information about the intervention and inviting reflection on community members’ expectations and level comfort with the initiative and research team’s readiness to work with the community is an important requirement in this phase.

The domains of information covered in each of the I-RREACH phases are summarized in Table 3. Using the identified key domains, a comprehensive tool was developed to gather information and initiate dialogue. The complete tool adapted to the hypertension management intervention in Aboriginal communities in Canada and rural villages in Tanzania as part of the DREAM-GLOBAL RCT is provided in Additional file 1. The tool can be easily adapted to support implementation of interventions for other health issues in various environments.

Gathering information on key implementation domains from different sources as suggested in the I-RREACH tool allows for rigorous data collection, including identification, triangulation and member checking of implementation issues. The collected information helps implementation teams to build on existing community strengths and anticipate and ameliorate barriers to program implementation. The rich qualitative data from key informants as well as the lived experience of community members allows researchers to gain a deeper understanding of practical implementation aspects and community expectations. This, in turn, allows researchers to reduce their knowledge gaps and respond to local needs more effectively. Further, by engaging in meaningful dialogue, it is hoped that unexpected miscommunications or errors made along the way will be similarly communicated within the team of academic and community members for quick resolution.

The I-RREACH tool was implemented during community visits in five Aboriginal Communities in Canada and in two communities in Tanzania. A total of 135 informants participated in twelve focus groups and seven interviews in Canada and Tanzania. Modest incentives were provided for participants and included a catered meal and small monetary compensation in recognition of participant’s time and incurred expenses. Translators were present as required. An interactive education session on hypertension tailored to the audience was offered prior to the focus groups.

After their participation in I-RREACH focus group or interview sessions, 83 participants completed an evaluation of the I-RREACH session. When asked to assess the content of the I-RREACH dialogue, most of these participants (98.8%) agreed or strongly agreed that the interview and focus group questions were clear, helped the researcher to understand their perspective (90.4%) and enhanced their understanding of the project (95.2%). From the perspective of cultural safety, 95.2% agreed or strongly agreed that they felt comfortable with the I-RREACH sessions and 97.6 % agreed or strongly agreed that the I-RREACH sessions were a good way to exchange ideas between the research team and the community stakeholders. Table 4 provides the evaluation tool and a summary of the relative scoring completed by participants in this study.

Based on an analysis of the written answers to open-ended questions in the evaluation, we also learned that the vast majority of participants were comfortable with what was discussed during the focus group sessions and also why people felt this way. The most common explanations provided included that they enjoyed learning about high blood pressure and its management and they appreciated the positive and respectful demeanor of the presenting research team members. This underscores the value of including engaging education activities in community visits. Many participants stated that the informal atmosphere made them feel comfortable and at ease. Many appreciated the opportunity to share their views and experiences of high blood pressure with others and to listen to others’ perspectives in the friendly and empowering atmosphere.

Most participants indicated that no changes for I-RREACH were needed; however, based on the content analysis of the open-ended questions, some stressed the importance of ongoing close contact between the research team and participants, requirements for training opportunities for staff and support for potential participants who struggle with healthy lifestyles due to difficult economic circumstances. Their recommendations are in line with the CBPR approach.

According to Buchannan [36], the most widely shared definitions of theory in health sciences are still based on the positivist ideas that a theory should be testable and generalizable. While this positivist model is adequate to explain and predict biological aspects of health problems, the complexity of human practices and actions and variability in the characteristics of health care settings makes a natural sciences model much less appropriate for health care research.

Researching not only if but also how an intervention has an impact on change on the other hand provides more useful knowledge and allows weak links in the causal chain to be strengthened during the implementation. The UK’s Medical Research Council’s guidelines [37] for developing and evaluating randomized controlled trials for complex interventions emphasize the importance of understanding how and why an intervention works.

“The rationale for a complex intervention, i.e. what changes are expected, and how change is to be achieved, may not be clear at the outset. If so, a vitally important early task is to develop a theoretical understanding of the likely process of change, by drawing on existing evidence and theory, supplemented if necessary by new primary research, for example interviews with stakeholders ”[23].

Similarly, Green argues that theory in health research should not be considered as “offering universal explanations or predictions, but rather as enhancing understanding of complex situations. Such understanding will inevitably need to be sensitive to specific contextual factors, and would necessarily draw on the experience of practitioners and communities” [38]. The notion of practice-based knowledge (or praxis) focuses on what we have learned through experience on how to move from theoretical knowledge to practice at the point of intervention [39]. Aristotle is credited with advancing the idea of praxis as knowledge which is based on experiences with social and historical relationships which correspond to “what we might call wisdom today” [36].

The notions of praxis and importance of community-held wisdom for implementation of a complex intervention became important theoretical underpinnings contributing to the development of the I-RREACH tool, as the team drew on researcher and community stakeholder experience to recognize the domains that may impact on the effectiveness of interventions in Aboriginal and Tanzanian communities. Furthermore, community stakeholders firmly emphasized the need for bidirectional learning, dialogue and ongoing reflection related to project implementation, which will provide ongoing information to the research team on how the intervention is working or why it is not. Similarly, Leykum and colleagues have advocated for the integration of local sensibilities and ongoing reflection into pragmatic research design [24]. Based on our experience thus far with DREAM-GLOBAL, practical reasoning can provide a critical lens to focus our inquiry to recognize and respond to insights based on the unique community and organizational context, circumstances and challenges as well as historical and cultural dimension that may affect the intervention.

In order to establish good working relationships, it is essential that university and community-based collaborators have an in-depth understanding of one another’s needs, resources and expectations related to research and action. Jacklin and Kinoshameg have argued that in order to be successful in CBPR, researchers “must unlearn the expert role they have been entrenched in” [40] and the I-RREACH tool can facilitate this process. Furthermore, implementation researchers require education not only by listening during the CBPR process but also more formally by learning about topics such as community history and cultural safety in research [41]. Therefore, research teams should also learn about and critically reflect on broader origins of ill health in their partner communities through the implementation process.

In Canada, there is considerable evidence that governmental assimilation policies, residential schools and forced changes of lifestyle imposed on Aboriginal people have resulted in much of the disease burden and health inequities in Aboriginal communities seen today [42-44]. In the past, these forms of oppression were touted as being in the best interest of Aboriginal people. Aboriginal perspectives were marginalized and negative outcomes were frequently associated with change imposed by outsiders. Given this negative impact of colonialism, Aboriginal communities often refuse standardized solutions offered or imposed by outsiders, and researchers must be cognizant of these historic facts. However, the consequences of a colonial legacy are often difficult to understand for researchers and representatives of governmental institutions, who are typically deeply entrenched in their own worldviews. Canadian Indigenous scholar Willie Ermine aptly observes that

“One of the festering irritants for Indigenous peoples, in their encounter with the West, is the brick wall of a deeply embedded belief and practice of Western universality. Central to the issue of universality is the dissemination of a singular world consciousness, a monoculture with a claim to one model of humanity and one model of society” [45].

The community stakeholders’ insistence in this study on focusing on a range of factors that may affect the outcomes of interventions beyond the community’s direct control was based on their practice-based knowledge as well as an understanding of the hegemony that Ermine describes. The unique influences that underlie health care barriers and disparities in Indigenous communities are complex and include key factors such as socioeconomic status, racism, cultural and communication differences, rural location [46] as well as related lack of resources for services and lack of incentives for providers to practice in rural areas. This perspective is also supported outside of the Indigenous health literature. For example, the notion that an intervention is affected by many factors beyond the control of the community is described by Tee and colleagues. They argue that even a strongly supported intervention within a health system can be seriously affected by “political, financial, educational, cultural, logistic, anthropological, and emotional” barriers [47]. Trickett and co-workers emphasize the importance of influence of culture, the state, policies and ideologies on community interventions [28].

The situation in communities in low-income countries is similar as shown in a recent systematic review indicating that health equity initiatives face potential barriers from a host of challenges in LMICs, including potential resistance from influential actors whose interests or values could be challenged by the intervention and social processes that exclude special interest groups such as rural communities from decision making or intended benefits of the intervention [48]. The review indicates that “politics, process and power must be integrated into the study of health policies and the practice of health system development”. Conversely, research in South Africa shows the positive impact of meaningfully involving local people including Community Health Workers (CHW) in community-based initiatives designed to develop interventions for hypertension [49].

Finally, our evaluation data supports the theoretical framework and provides preliminary support for the validity of the I-RREACH tool as an appropriate implementation tool from the perspective of community stakeholders. The evaluation data also provided important feedback to the research team on the effectiveness of their approach and their own increasing level of cultural competence and safety. We therefore recommend that the evaluation form be used whenever the I-RREACH tool or an adapted version of it is used. Participants should be provided with several minutes to fill out the form (see Table 2) anonymously and to return forms in blank envelopes in order to reduce the social desirability bias. If participants’ literacy is not sufficient to fill out the form, community helpers are needed to help fill out the forms based on the verbal feedback of each participant (as our team did in Tanzania).

The I-RREACH tool has been successfully used to support the implementation of the DREAM-GLOBAL pragmatic RCT and the related health services changes with formal positive feedback from participants as described in this paper. The DREAM-GLOBAL study is still in its implementation phase, and comprehensive data on recruitment and retention of participating communities, providers and patients is not yet available. Additional time and research is required (1) to conduct a detailed process evaluation to inform best practices for the implementation of the tool; (2) to document how the I-RREACH tool and process enabled understanding of implementation issues that can foster effective relationships between researchers and community stakeholders throughout the span of a project ; and (3) to analyse the specific collaborative, CBPR and inter-professional skills required of the research team members to successfully implement I-RREACH.