Developing primary care teams prepared to improve quality: a mixed-methods evaluation and lessons learned from implementing a microsystems approach

At the launch of the redesign initiative in 2008, the academic health center consisted of three separate, legal organizations: UW School of Medicine and Public Health, UW Medical Foundation, and UW Hospital and Clinics. Clinics were owned and operated by one of these organizations but were collectively branded as UW Health. Forty-six primary care clinics were located throughout the state of Wisconsin, with the majority in the Madison area. There were 276 primary care physicians: 38 pediatricians, 62 general internists, and 156 family physicians who conducted ~ 649,000 primary care visits annually. Practices varied in size from 1 to 20 physicians.

The three chief executives launched the UW Health Primary Care Redesign initiative, an ambitious effort to transform delivery of primary care within the entire system regardless of clinic location, primary care specialty or ownership [23]. The need to radically transform care delivery arose from access issues within primary care and lagging clinical quality performance compared to other health systems in the state as publicly reported by the Wisconsin Collaborative for Healthcare Quality, a voluntary performance reporting system (www.​wchq.​org). These issues reached a critical state when a new provider group entered the local market and successfully recruited several faculty physicians. This acute loss of established physicians coupled with the national difficulty of attracting new physicians to primary care created a short term access crisis that prompted a focus on transforming the primary care system to achieve what is now termed the quadruple aim [24].

The initiative was guided by evidence-based principles for transformative change [25‐27]. UW Health leaders in quality and safety had developed a framework for change based on redesign principles from the Institute of Medicine (IOM) and organizational behavior theory that was used to organize its large-scale change initiatives [28]. Kotter’s 8-steps for organizational transformation provided guidance for change over time [25]. A governance structure, the Primary Care Steering Committee, was created consisting of physicians and administrative leaders responsible for primary care. A vision for the transformed primary care system was created by a broad representation of front-line clinicians and staff working collaboratively with senior leadership and communicated widely throughout the organization. The academic health system’s Health Innovation Program was created to integrate healthcare research and practice. In support of this mission, health services researchers were included on the Initiative’s strategy oversight and implementation teams. Researchers led the study design for the evaluation including collection of qualitative data that was rapidly analyzed and shared with operational leaders in ‘near real time,’ providing important findings that informed future implementation strategies.

The Dartmouth Clinical Microsystem curriculum was considered by the Steering Committee to be the framework that best fit existing organizational strengths in process improvement education and its readiness to address change at all levels of the health system. Prior to implementation of this approach, primary care clinic leaders had participated in structured process improvement education incorporating concepts from various models including the Institute for Healthcare Improvement (IHI) model of improvement, systems thinking, and Lean Six Sigma concepts [29‐31]. The clinical microsystem approach specifically was selected because it allowed for those who were affected by the change to design the change by equipping front-line care team members and administrative managers with common team-building and improvement skills useful for redesigning care processes.

In this approach, a microsystem is defined as a small care unit consisting of a care team, its panel of patients, and the core processes that produce its patterns and norms [4]. Within the organization, a primary-care microsystem generally consisted of a physician (or several part-time physicians), a receptionist, registered nurse and other clinical staff (e.g. medical assistant, licensed practical nurse) and often included an advanced practice provider and residents. Figure 1 depicts these organizational levels.

Table 1 provides a summary of program goals, components and involvement. As shown, program activities consisted cohort learning sessions, team meetings, and practice coaching. During these activities teams learned how to continuously assess their microsystems. Data analysis and team input drove the establishment of improvement aims and implementation of interventions. Initially aims could be developed in any area, but by the third cohort microsystem teams were asked to focus improvement efforts on specific areas (e.g. access) that were aligned with organizational priorities and supported by organizational resources.

Coaches provided ongoing teaching on how to follow the FOCUS (Find-Organize-Clarify-Understand-Select) PDCA (Plan-Do-Check-Act) improvement process and apply appropriate improvement tools. The coach also helped facilitate team building, team communication and patient engagement. After 6 months, coaches met their teams with decreasing frequency to reinforce skills and by 12 months transitioned contact to an as-needed basis.

The overall objectives of this concurrent mixed-methods evaluation was to understand: 1) the short term and sustained impact of the program on team effectiveness and cohesion, quality improvement skills, and communication skills; 2) What challenges occurred during program implementation and were these able to be overcome? The multidisciplinary evaluation team consisted of physicians and staff skilled in quality improvement methodologies, mixed and qualitative research methods, programs evaluation, and statistics. Consistent with common use of mixed-methods in dissemination and implementation science [32], for our first question we used a quantitative method, a longitudinal survey, to examine deductively the outcomes of the implementation with breadth across team member participants. For our second question, we employed multiple qualitative methods, individual and group interviews/discussions, to examine inductively with depth the implementation context and process by sampling multiple participants and implementers during the first two cohorts (when significant challenges were expected to be discovered), and program leaders across the entire implementation period. Individual interviews were used for participants to enhance confidentiality, and group interviews were used for implementers and leaders in order to capture the range of opinions and agreement. These concurrent methods received equal weight (QUAN + QUAL) and were mixed for complementarity. Throughout the study period because our research was closely embedded within the implementation process, we used an empowerment evaluation approach [33], which aims to aid organizations in improving their innovation during the implementation process through sharing and discussing data and encouraging reflection. Therefore, during the program we collected and reported to program leadership findings about challenges from the qualitative data near “real time” and reported the analyzed quantitative outcome data every 6 months. As described further below, the summative evaluation in this paper presents the quantitative longitudinal outcomes, context and resolved and ongoing challenges analyzed from the qualitative data across the multi-year program.

Quantitative evaluation consisted of an electronic survey sent to all team member participants at baseline, 6, and 12 months. Across all cohorts and time periods, 15 items assessed perceptions in the following domains: organizational support, team cohesion and effectiveness, quality improvement and meeting skills, and communication effectiveness. A single question assessed overall satisfaction with the program at 6 and 12 months. Responses were generally on a five-item Likert scale (strongly agree, agree, neutral, disagree, strongly disagree). Two survey items about patient engagement were analyzed for a separate publication [34] and thus these findings are not presented. Several survey items were taken from prior Microsystems tools [28, 35] while others were created or adapted for the local context. These adaptations reflected the local purpose or specific leadership roles and structures (e.g. “to make my work meaningful” adapted to “to be effective in my job”; “clinic manager support”; “physician leader communication”).

These data were augmented by individual interviews, focus groups and facilitated stakeholder group discussions led by the first author or a masters-level researcher trained in qualitative methods. Interviews were generally conducted on-site and lasted 15–75 min. During the first and second cohort (2009–2010), 69 team members from 11/17 teams were interviewed. Program leaders informed the selection of these teams using a maximum variation approach by varying primary care specialty, location (rural or urban), and degree of engagement in program activities. All team members who were present on the days that the research team was on-site were interviewed. Because teams microsystems varied in membership, interviewees came from a variety of roles: 14 physicians (3 residents), 6 Advanced Practice Providers, 8 Administrative supervisors, 1 Pharmacist, 7 Registered or Licensed Practical Nurses, 17 Medical Assistants/Certified Nursing Assistants, 4 Schedulers, 4 Radiology, Laboratory or Medical Technicians, and 8 Receptionists. Additionally, two one-hour clinic manager and program leader focus groups and one coach focus group were held. Lastly, four longitudinal, facilitated structured discussions about ongoing lessons learned were held with stakeholders responsible for program development, implementation, and evaluation. Six participants requested that their interview not be recorded. In these cases, handwritten notes were taken during the interview and narrative summaries were constructed immediately afterwards. For the remaining 63 interviews, all focus groups, and structured discussions, audiotaping and transcription occurred.

Qualitative analyses employed an immersion/crystallization approach [36]. Two individuals skilled in qualitative analysis independently read all transcripts and summaries, understanding each clinic team as a case and paying particular attention to any text that identified implementation challenges and solutions. This text was transferred to a structured matrix in Excel that was organized by both clinic site, cohort, and role. Text was examined line by line and coded to identify the type of challenge (e.g. time, knowledge, technology, motivation, spread) and attempted solution. These codes were aggregated into themes by topic (e.g staffing and support, participation motivators, sustainment), and the researchers met weekly to discuss these themes. Because of the participatory aspect of the evaluation, researchers also triangulated quantitative findings with the qualitative themes that emerged across the different levels of data by role (individual team members, managers, coaches, and program leadership). This allowed program implementers to make adaptations to the intervention and involve those that were experiencing the challenge. Member checking occurred through regular public presentation of findings back to participating individuals and leadership as part of the participatory approach to the evaluation.

204/257 (79%) of individuals completed the baseline survey, with responses from all 58 teams. Across cohorts of teams receiving Microsystems training, response rates ranged from 49 to 92% at baseline, 45–77% at 6 months, and 52–81% at 12 months. The mean (standard deviation) response to the question “participating in the Microsystems program has been a worthwhile experience” was 3.83 (0.91) at 6 months and 3.80 (0.90) at 12 months, with responses on a scale of 5 = strongly agree, 4 = agree, 3 = neutral, 2 = disagree, 1 = strongly disagree.

Table 3 displays longitudinal aggregate mean responses to the remaining questions organized by domain. In terms of organizational support, respondents did not perceive a significant change in their clinic manager’s support of their microsystems work (4.10 (mean), baseline (time point); 4.21 6 months; 4.25, 12 months). At 12 months compared to baseline, there were statistically significant increases in microsystem team member’s perceptions of having the tools and equipment needed to be effective (3.32, baseline; 3.50, 6 months; 3.66, 12 months) and useful knowledge and information about their patient population (3.38, baseline; 3.69, 6 months; 3.74, 12 months).

Respondents perceived high team effectiveness and cohesion from the outset of the program. Two items showed statistically significant improvements at 6 months—the ability to solve difficult tasks if effort was invested (3.94, baseline; 4.19, 6 months) and having a strong bond with team members (3.90, baseline; 4.16, 6 months). Items that did not show significant changes queried about group capability to perform tasks well, manage unexpected troubles, and competency to make team level improvements.

Also as shown in Table 3, responses indicate that team members perceived significant gains over baseline at 6 months and 12 months in all five quality improvement skills queried: knowing the top processes needing improvement, using data to identify improvement activities, creating a focused meeting agenda, serving in different team roles and knowing the purpose of the team. Respondents also perceived significant gains in effectively communicating their work to the manager (3.13, baseline; 4.01, 6 months; 4.00, 12 months) and other teams in the practice (2.96, baseline; 3.48, 6 months; 3.40, 12 months).

Several key challenges to the success of the program were identified by teams and clinic leadership during the implementation process. For many of these, the organization was able to respond to Microsystem program needs and develop specific solutions; Table 4 describes challenges where a solution was implemented, along with a representative quote. These included: lack of time and coverage for program participation, need for technical/technology support, lack of perceived value for improvement work, and difficulty spreading learnings.

However, additional challenges persisted. These challenges are described below including: tensions between team-and clinic-level change, part-time workforce, team instability, challenge of a structured QI approach, and difficulties aggregating learnings.

From the survey findings, team members perceived the program provided them with additional support- data, knowledge, equipment and improved communication. The in-depth interview findings did not contradict these findings. However, leaders did not measure the success of the program according to these metrics. Instead, the focus was on resource use, rapidity of change, and scalability.