Understanding how context and culture in six communities can shape implementation of a complex intervention: a comparative case study

The implementation of Health TAPESTRY was facilitated by an academic family medicine department and conducted within six communities of varying sizes across Ontario, Canada. This implementation of the program was a scaling-up of the earlier implementation of Health TAPESTRY during the initial effectiveness trial [6]. Each community was associated with a family health team (FHT). FHTs are physician-led primary care practices with embedded interprofessional care providers and are a common model for providing primary care in the province [23]. As the intervention also included community volunteers conducting home visits to older adult clients, the setting also included client homes in the six communities, as well as the organizations that coordinated the volunteers’ work. Four of the communities were supported by a national humanitarian charitable organization while the other two were supported by a coalition of agencies focused on community health.

In Health TAPESTRY, two trained volunteers visit older adult clients in their homes and use structured surveys to ask about clients’ health needs, goals, and social context [5]. Client responses are entered into a web-based application (the TAP-App) using a tablet computer. A summary (the TAP-Report) is sent to the client’s primary care team where a plan of care is created and implemented by a huddle team, a small group of interprofessional health care team members. After six months, the volunteers return to clients’ homes to repeat the surveys and determine if clients’ goals were met. Each community has a huddle lead who facilitates the huddle and a physician champion who is an advocate for the program and actively involved. FHTs can choose to provide clients with a client-friendly TAP-Report which is a brief summary of their survey results and a written explanation of the plan of care. They can also choose to send volunteers back within the six-months to do any necessary follow-up.

This qualitative case study takes inspiration in its design from Bartlett and Vavrus’s comparative case study (CCS) methodology. CCS describes culture and context more broadly than in many case study methodologies and we will use those definitions in this paper. According to CCS, culture can be defined as the development of sense-making processes rather than a single static ‘culture’; context can be understood beyond geographical boundaries or a strictly bounded case and defined more fully as interconnectedness with surroundings and hierarchies [24]. Specifically in our project, when we talk about culture we are talking about the personal, relationship-based, and organizational culture of the individuals involved in implementing Health TAPESTRY and context as interconnectedness between these people, their organizations, and beyond.

We compared our six cases (the six Ontario communities implementing Health TAPESTRY) using a process-oriented approach to make sense of implementation [24]. When we talk about implementation in this paper, we mean how communities managed to carry out (i.e., implement) the Health TAPESTRY intervention. To understand this concept, we focused on perspectives of our multiple stakeholder groups on what is working well (i.e., facilitators of implementation) and what is not working well (i.e., barriers to implementation). We also incorporated considerations of power structures and relations, horizontal comparisons between communities, and vertical and longitudinal feedback about hierarchy and context where available [24]. In this paper, we chose to focus on aspects of the cases that had distinctions between communities, rather than those that were common between them.

Focus groups, interviews, and narratives from key informants on the research team were used to describe and compare the cases. We conducted separate focus groups in each community for each of three stakeholder groups (members of the huddle team, providers outside the huddle, and volunteers). We conducted semi-structured interviews with clients, clinic managers, volunteer coordinators, and other key providers or volunteers who could not make the focus group time. All members involved with Health TAPESTRY (listed above) in each site were invited to participate in either a focus group or interview by email, except for the clients. We used convenience sampling stratified by site to invite intervention clients by telephone to participate in an interview. Implementation was in a rolling fashion; however, data collection across each community had the same timeline: client interviews were conducted once clients had finished their 6-month volunteer visit, volunteer and volunteer coordinator data were collected one year after volunteers had first visited clients in that community, and clinic team members’ data were collected after ten 6-month reports had been seen in that community. All focus groups were held in the participants’ communities at a primary care clinic (with at least one of the two facilitators in person), and most interviews were held over the phone, with a few in person if a facilitator was visiting the participant’s site (e.g., with some clinical managers or volunteer coordinators).

Focus groups were facilitated at a community location by two research team members, at least one of whom was in-person with participants. Interviews had one facilitator and were either in person or over the phone, depending on participant availability and convenience. Facilitators included: HB, RC, JD, SD, JG, CK, and FP. The focus groups and interviews were semi-structured, using a question guide informed by the interview guide from the previous implementation of Health TAPESTRY, and slightly adapted for each participant group, with a focus on program improvement. Interviews and focus groups were audio recorded and transcribed. Facilitators also made field notes.

Key implementers of Health TAPESTRY in the research team reviewed the data that were collected and added an understanding of the implementation across the six communities including the context, culture, and hierarchies inherent in each case (i.e., community).

Transcripts of focus groups and interviews were uploaded into NVivo 12 [25]. Three researchers with experience in qualitative research and Health TAPESTRY (SD, JG, and FP) coded and analyzed the transcripts. The three coders all identify as female, have graduate degrees, may have known some of the participants in the interviews and focus groups, and had comprehensive knowledge about Health TAPESTRY as they were both implementors and evaluators of the program. We followed the six phases of thematic analysis described in Braun and Clarke [26, 27] as described below, but coded at a more semantic rather than a reflexive or latent level. First, JG and FP independently familiarized themselves with the data (phase 1), and then jointly created a basic coding structure based on our interview questions. Initial transcripts were coded deductively (phase 2) based on the question guide (Additional File 1), i.e., the initial categories within the codebook were based on the key questions we asked in the interviews and focus groups, which in turn were the general areas we wanted to probe in order to understand participants’ perspectives on Health TAPESTRY and its implementation (e.g., what was working well and what was not working well with the program). This was based on our previous program implementation and our program evaluation needs in this round. The remaining coding was generated more inductively, adding further codes and categories beyond the basic categories already included. All transcripts were independently coded by one of the coders and then checked by another coder to ensure consistency. We then began searching for (phase 3) and reviewing themes and re-organizing codes into higher level categories and themes (phase 4) and then naming the themes (phase 5). Any disagreements were resolved through regular discussion and meetings between analysts (with the occasional inclusion of RV). Phase 6 of thematic analysis is creating a report; the report is this manuscript.

After the preparation of the NVivo database including focus group and interview data from all sources, we split cases (i.e., the six Ontario communities implementing the program) and compared them on all elements of the Health TAPESTRY model via the matrices function. There is a long history of the use of matrices in qualitative research [28]. Converting the textual qualitative data to numbers, often referred to as “quantitizing” in the mixed methods literature [29‐31], can help researchers identify patterns in the data, clarify meaning in the data, contribute to the display of data, and help readers understand and interpret the data [28, 32, 33]. We used intensity matrices, where cell contents are numerical and higher numbers indicate higher intensities of frequencies [28], and we did this on a case-by-variable matrix [29]. Using the matrices produced by NVivo in this way allowed our research team to visualize the proportions of frequencies by community across themes. While the Health TAPESTRY program as a whole has multiple important elements, stakeholders, and constructs, to compare between sites we wanted to compare only the elements that showed differences. JD, JG, and LL reviewed, compared, and discussed the matrices to understand which aspects were distinctive between communities. It was at this point that we determined that while several categories and themes were very similar across communities, three elements had differences between communities: 1) interprofessional teams’ patterns of work, 2) volunteer program coordination, and 3) the client experience.

Once reviewing all matrices, we chose to focus on the description of three key areas of Health TAPESTRY which were the most distinct between communities: the work of interprofessional primary care teams, volunteer program coordination, and the client experience; incidentally, these also represented the three key stakeholder groups in the dataset. The percentages within the matrices are indicative of the frequency of each theme (row) within a category by community. Afterwards, these researchers provided narratives about additional contextual factors that may have led to these differences, as well as potential implications that would not have been shown in the qualitative data alone (Table 2).

Enhancing qualitative rigour was considered in varied ways. Credibility and confirmability were enhanced by using multiple data sources (interviews, focus groups), multiple perspectives (clients, health care team members, volunteer coordinators, and volunteers), and multiple analysts (JD, SD, FP, JG, and LL) [24, 34]. We used the COREQ checklist to guide our reporting of this study (Additional File 2). The potential generation of theoretical insights that could help understand cases beyond the ones described in this paper were enhanced through thick description in settings and cases [24].

Health TAPESTRY has four key parts that have been identified in previous published work: 1) trained community volunteers who meet with clients in their homes and gather health and social information; 2) the use of technology for collecting and sharing information between clients, volunteers, and the health care team; 3) interprofessional primary health care teams who support clients with their health goals and needs; and 4) community engagement and connections; all of these parts encircle the client [5, 6, 35]. Based on the previous implementation and evaluation of the program, including implementer, participant, and stakeholder consultation, these have been identified as the core intervention components. The “core component” literature understands this term to mean the essential functions, principles, activities, or elements that are needed to produce the desired elements, i.e., what elements produce a potentially effective program [36]. While every community included of these key elements, there were distinctions even between some of these core components when the program was adopted and subsequently adapted by each of the six community. It is important to understand context differences when an intervention is implemented in a new setting [37]. Based on the qualitative matrix exercise, there were distinct differences between communities in the areas of interprofessional teams, the volunteer program, and the client experience. Beyond deepening our understanding of the core components of Health TAPESTRY that had distinctions between communities, we also structured this paper including the first two domains of the Model for Adaptation Design and Impact (MADI): 1) adaptation characteristics; and 2) possible mediating or moderating factors, which we include thoughts about in the Discussion [37]. The third domain of MADI is implementation and intervention outcomes, but this evaluation was not designed to identify or compare outcomes [37].

In total 135 people participated in either an interview or a focus group. No one explicitly refused to participate or withdrew from the study. Focus groups were about one hour long and interviews were approximately 30 min. Our sample had representation from all participant groups from each of the six communities (Table 1) and our sample size per site aligns with the expected number of people needed to meet saturation based on the previous implementation of Health TAPESTRY.

The six communities that Health TAPESTRY was implemented in ranged from a small town to a large city. The participating FHTs differed in their roster size, number of physicians, number and assortment of allied health professionals, and the services the clinic offered. One FHT consisted of two separate clinics, each with their own huddle teams. Table 2 describes three key areas relating to implementation and the flow of information throughout the Health TAPESTRY program for each community. Table 2 also maps to Domain 1 of MADI: adaptation characteristics; this domain asks what was modified, the nature of the adaptation, when did the adaptation occur, for whom/what was the adaptation made (i.e., the reason), and who participated in the decision [37].

The huddle, volunteer program, and client experience elements in Table 1 describe what was modified and the nature of that adaptation, along with when it occurred (if during implementation – unstated times occurred prior to program implementation in that community). In each community, the reason for each of these adaptation differences was made largely based on feasibility within their unique setting. Who made these decisions? The key personnel from the interprofessional primary care team made most of the decisions (including those related to their team, client communication, or volunteer connections with their team), supported by the overall Health TAPESTRY program manager who ensured the changes did not stray from the core components; the volunteer coordinator both made decisions and took action regarding volunteer training.

The following describes the saliency of the themes within the three key areas of difference we found and chose to examine: interprofessional teams’ patterns of work, volunteer program coordination, and client experience.