Assessing the implementation processes of a large-scale, multi-year quality improvement initiative: survey of health care providers

The province-wide introduction of Lean as a strategy to improve health care quality and promote patient-centredness was initially launched by the Government of Saskatchewan of Canada in 2012 [1]. Lean refers to a set of operating philosophies, tools and activities that help to create maximum value for patients by reducing sources of waste in a process [2, 3].This large scale transformation was aimed at effecting coordinated, system-wide change affecting multiple organizations and care providers [4]. In keeping with recommendations that active implementation strategies are essential to producing a coherent and multilevel approach to health care transformation [5‐8], significant investments in implementation were directed towards training initiatives, particularly at the senior leadership level. Initial implementation of Lean was delivered through a paid consultant and later transitioned to local leadership. More than 1000 projects [1] have been undertaken throughout the publicly-funded health regions in this province of 1.1 million people and the government re-confirmed its commitment to support the ongoing use of Lean in its 2016 strategic plan [1].

Because Lean is a “high-touch, high-maintenance enterprise” [9, 10], the costs of implementation are high. Estimates of the investment by the provincial government in Lean since 2010 are in excess of $44 million dollars [11], contributing to the politicization of this quality improvement strategy [12] in a setting where public tax revenues fund the health care system. The Provincial Auditor has noted the absence of evaluation on the outcomes of Lean implementation [13], although the government reported that the initial outcomes of Lean implementation have been promising. Evidence supporting the use of Lean in health care, however, is weak, with a recent systematic review [14] concluding that Lean interventions: were not associated with patient satisfaction or health outcomes; were negatively associated with financial costs and worker satisfaction; and had potential but inconsistent benefits for safety and patient flow.

Achieving the stated goals of Lean implementation to improve health care quality and promote patient-centredness is contingent upon health care professionals working individually and collectively to embed this new strategy into their organizational and professional contexts [15]. Given the lack of consensus about how to measure the success of complex interventions in general [15, 16], Normalization Process Theory (NPT) [17] offers an approach to understanding the complex influences that make innovations such as the introduction of Lean “workable” in practice settings [18, 19]. The overall aim of this study was to examine the extent to which Lean has become embedded within health care after four years of implementation using an online survey of licensed health care providers. The objectives of the study were to: a) evaluate the implementation processes associated with Lean implementation in the Saskatchewan health care system from the perspectives of health care professionals; and b) identify demographic, training and role variables associated with normalization of Lean.

Although there is a substantial literature relating to implementation of complex interventions in service organizations, understanding the ways in which large-scale transformations are embedded and integrated into evolving and diverse practice settings remains a challenge.⁸ Normalization Process Theory (NPT) [17] is one approach to better understanding the work required at individual and collective levels to implement complex interventions. NPT addresses the generative processes that underpin: a) implementation (bringing practices into action); b) embedding (when practices are routinely embedded into the everyday work of individuals and groups); and c) integration (when practices are reproduced and sustained within the social matrices of an organization). Four core constructs have been identified that represent the different kinds of work people and organizations do to implement a new practice such as Lean; each construct is comprised of four distinct components (Additional file 1: Table S1).

According to the NPT model, the organizing structures and social norms of practice environments influence the way in which new practices are accommodated and directly affect the organizing factors of a practice, while the group process and conventions affect the way in which a practice is produced and reproduced in actual patterns of interpersonal behavior [17]. Organizing structure and social norms, together with group processes and conventions, affect perceptions of coherence (the meaningful qualities of a practices) of an intervention to those involved in implementation. Coherence affects and is affected by cognitive participation (the enrolment and engagement of individuals and groups), which has a reciprocal relationship with the practices already existing (collective action). Reflexive monitoring refers to the ways a practice is understood and assessed by those involved. This understanding and assessment serves to modify both organizing structures and social norms as well as group processes and conventions.

A substantial body of evidence supports NPT as a means of explaining processes associated with implementation of complex interventions ranging from telecare and e-health [20, 21] to maternity services [22] to management of depression [23, 24]. Recently, the NoMAD instrument was developed to operationalize and measure the core constructs of NPT with the intent that this tool could be customized for a wide range of purposes and across different settings [15].

A total of 1378 health care providers completed the survey. Response rate by professional group is reported in Additional file 2: Table S2. Because there were only 35 physician responses, the decision was made to exclude their responses from the analysis. Participants’ profession was not disclosed in 308 surveys, of which 299 had at least 25% missing values to the NoMAD items, leaving nine surveys to enter the study. These nine subjects, along with three more with similar level of missing responses, were excluded from the analysis. On the basis of similarities in role, the various classifications of nurses were combined into one category (Nursing), while the remaining providers were aggregated into a second category labelled health professionals (HP). Responses were retained for: 734 nurses (395 Registered Nurses; 314 Licensed Practical Nurses; 25 Registered Psychiatric Nurses) and 298 Health Professionals (44 Registered Dietitians; 67 Physiotherapists; 56 Occupational Therapists; 131 Pharmacists). Details of the sample selection process has been summarized in a flow-diagram in Fig. 1.

Reflecting the composition of the healthcare workforce in this province [29], most respondents were female and direct care providers. Nurse respondents were older, less likely to work full-time and more likely to work in acute care settings than HPs. There were no significant differences between nurses and HPs in terms of years of experience, primary role or geographic location, with respondents from settings outside of the large urban centers being well-represented (44.1%). A full description of respondent characteristics is located in Table 1.

In terms of Lean familiarity, integration and the impact on workload (Table 2), wide variability was reported within both groups. While approximately two-thirds of respondents reported no, little or moderate familiarity with Lean principles and activities, 32.6% self-rated their familiarity as substantial or complete. Lean was not currently a part of their work for more than one-third of respondents, and 32% of nurses and 26% of HPS believed that Lean was likely to become part of their normal work in the future. Significantly different (p < 0.0001) impacts on workload were reported by nurses and HPs, with 75% of nurses and 42% of HPs reporting that Lean had increased their workload.

Participation in Lean training and activities is reported in Table 3. No significant differences in Lean training were noted between nurses and HPs. While the majority of respondents had attended an introductory session (Kaizen Basics), 17.5% reported they had received no training at all in Lean principles or activities. A separate cross-tabulation (not shown) to examine possible collinearity between reporting a primary work role as a senior or front-line leader and completion of Lean leadership training indicated that close to 70% of leaders had not received this specialized training. The majority of respondents (91%) had participated in at least one Lean activity. The Lean activity most frequently reported by respondents was the Visibility Wall/Wall Walk (71–73%).

Additional file 3 Table S3 displays responses to the NoMAD construct items. Because further psychometric testing is still required on subscale and cumulative scores, items are reported individually under the constructs and not tallied. Significant differences (p < 0.05 or greater) in responses were noted between nurses and HPs on 13 of the 20 items, with nurses displaying less propensity towards normalization in all cases.

Results for the multivariate multinomial regression models for each of the five selected outcome variables are reported in Tables 4, 5, 6, 7 and 8. Professional characteristics were significantly associated with agreement with the NPT items used as measures of Lean normalization. Having a leadership role, compared to a direct care role, was one of the strongest predictor variables for the cognitive participation, coherence, and reflexive monitoring models. Respondents in leadership roles were significantly more likely to agree than direct care providers with the following survey items: I support the use of Lean in health care (O.R. = 10.29, 95% C.I. 5.65–24.7), I can see the potential value of Lean for my work (O.R = 7.83, 95% C.I. 4.29–17.8) and that the people they worked with believed Lean was worthwhile (O.R. = 4.33, 95% C.I. 2.24–9.74). There were no statistically significant differences between those in a leadership role and those in direct care in terms of responses to the sufficient training or sufficient resources (collective action) items.

Professional designation as an HP, in contrast to being a nurse, was also associated with agreement responses to items representing cognitive participation, coherence, reflexive monitoring and collective action (training). HPs were significantly more likely than nurses to support the use of Lean in health care (O.R. =4.11, 95% C.I. 2.70–7.20); to see the potential value of Lean for their work (O.R. =2.57, 95% C.I. 1.73–4.12); to indicate that the people they worked with believe that Lean is worthwhile (O.R. =2.06, 95% C.I. 1.21–3.96); and to believe sufficient training had been provided (O.R. =1.57, 95% CI 1.02–2.45). No differences were identified between HPs and nurses in terms of the sufficiency of resources.

In terms of personal and contextual characteristics, increased workload was strongly associated with weaker normalization across all five models. The associations were particularly negative between somewhat increased workload (O.R. =0.19, 95% C.I. 0.11–0.29) or substantially increased workload and support for the use of Lean (O.R. = 0.05, 95% C.I 0.02–0.09). There were no significant associations between age group, years of professional experience, or employment status and any of the NPT outcome items. Weak but significant associations were noted between several normalization outcomes and: a) practice location (urban vs. rural/remote); and b) employment in long-term care.

More than four years after the ambitious and costly launch of Lean as a province-wide strategy to improve health care quality and promote patient-centredness within healthcare in Saskatchewan, the findings of our survey reveal that major gaps remain in embedding the principles and activities of Lean into everyday health care practice, particularly among nurses and direct care providers. Assessing implementation processes of initiatives such as Lean that are ongoing can serve to target issues that require additional support and attention.

Significant differences in responses were evident between: leaders vs. direct care providers; nurses vs. other health professionals; and providers who reported increased workload as a result of Lean vs. those who did not. Respondents in leadership positions were much more likely to view the implementation and benefits of Lean in a positive manner than were direct care providers. More than 75% of respondents indicated that neither sufficient training nor resources (collective action) had been made available to them for the implementation of Lean. Compared to other health care providers, nurses were more likely to report that Lean increased their workload.

The importance of strong leadership in effecting the cultural changes needed for major health reform has been well-recognized [30]. A wide gulf between the perspectives of leaders and direct care providers was apparent, however, with respect to the constructs of coherence, cognitive participation and reflexive monitoring. Based on the premise that senior leaders “are central to ensuring that Lean will pervade the organization’s management system and [to] provide an example of Lean principles to other personnel” [2], major investments had been made by the Ministry of Health to ensure that those in health leadership roles were “committed to learning and applying Lean principles” [2]. These investments at the senior leadership level included a rigorous certification program that included didactic and experiential training, including leadership of Lean events, and an opportunity to visit organizations in the U.S. that have long experience with the use of Lean in manufacturing and health care [31]. Given that only 30% of leaders had Lean Leader training, the basis for the strong support shown by leaders for Lean implementation might be better explained by factors other than training alone. A subsequent paper will compare the narrative responses on the survey between leaders and nurses to shed further light on this finding.

Adequate training and resources are foundational to any attempt to transform health care [32], but the majority of respondents believed these had not been sufficient (Additional file 2: Table S2). Only one quarter of all respondents believed that training had been sufficient and 18% felt that resources had been sufficient to implement Lean. The nature and type of Lean education and participation in Lean activities had interesting implications for the extent of normalization. The majority of respondents reported attending some form of Lean training, but only the variable of having no Lean training at all was associated with (lack of) support for Lean. Kaizen Basics training, which was originally an eight hour introductory program (later scaled back), was intended to provide staff with “a broad overview of Lean principles and methods, so they understand the changes taking place and have a sense of what to expect when they are invited to participate in an improvement event” [33]. This training was attended by three quarters of respondents, but we found no associations between attendance at these sessions and the normalization outcomes examined in this study.

Participation in Lean activities was not associated with the items representing normalization of the Lean approach, in spite of the appeal of providing “hands-on training” as a pedagogical strategy. Our findings contrast with those reported in a recent study examining the introduction of Lean in primary care that noted that the time and intensity of exposure to redesign activities was a positive influence on acceptance by providers [34].

The anticipated benefits of the implementation of Lean are contingent upon widespread dissemination of the philosophy and processes throughout the entire system, resulting in a major cultural shift that focuses on continuous performance improvement [32]. Critical factors such as perceived lack of ownership and subcultural diversity have been noted to derail attempts at transforming culture when direct care providers are not engaged [12]. As recently noted by Bohmer [35], “delivery of care is ultimately governed by structure and process at the ward, clinic or practice level”. Lean implementation in health care can be perceived to compromise professional autonomy [36] and can challenge established professional hierarchy [12], thereby promoting resistance from key stakeholders required to enact the transformation within the system. McIntosh [12] noted that the Saskatchewan government failed to appreciate that key factors such as nurses (and physicians) had the independent ability to push back against this initiative in response to top-down implementation and that Lean “cannot be implemented in the top-down, directive manner displayed to date but must accommodate other powerful decision-makers in these sectors” [12].

Although nurses constitute the largest proportion of licensed health care providers in health care system and their contributions are well-recognized to be pivotal in ensuring patient safety and high-quality care [37], the perspectives of nurse respondents suggest that there are fundamental differences in the way in which Lean impacts the work of nurses compared to other health professionals, despite similarities in Lean training and participation in activities. Nurses play many critical roles in health care delivery, but globally share concerns about understaffing and inadequate training and support [38]. Nurses have been found to encounter an average of 8.4 work system failures in an 8-h shift, a fact compounded by frequent interruptions of their work [39, 40]. Nurses, compared to physicians, have been found to have less work autonomy, fewer professional development opportunities and fewer options for career change [41]; nurses are considered the health professionals most exposed to work strain that compromises their physical and psychological well-being [41, 42]. The implications of Lean for nurses’ work, particularly for bedside nurses, requires further thoughtful consideration. A 2014 survey conducted by the Saskatchewan Union of Nurses [43] reported a statistically significant negative effect of Lean on nurse engagement, usefulness, patient care, time for patient care, workplace issues, availability of supplies, workload, stress and patient care. Our results lend support to the notion of a potential misalignment between the principles and activities of Lean as it had been implemented and the work of clinical nursing, which we are currently investigating in related projects.

Our findings also point to the importance of context in the implementation of Lean. Greater agreement with normalization constructs was noted for non-urban respondents, whose practice involves different demands than larger urban settings and highlights again the importance of context in introducing system-wide change. Context is a critical consideration in the implementation of large-scale interventions [44‐47], with “dynamic elements of context play[ing] a powerful role in shaping participants’ capacity and potential” [46] Complex interventions that cannot be integrated smoothly within organizational contexts and do not prove to be workable alongside other tasks and duties are unlikely to be normalized [48]. Conversely, adaptations to organizational contexts and tasks and duties may act to facilitate the normalization of interventions such as Lean.

Our results highlight the importance of attending to increased job demands that may result from new interventions. Job demands are aspects of work associated with physical, emotional and cognitive efforts and can translate into job stressors and burn-out if sufficient resources are not available [48]. Perceived increased workload was the only variable to be a significant predictor of negative responses across all five of the models. The implementation of Lean was reported to have different implications for the workload of individual respondents, pointing to the need for leaders to carefully evaluate how interventions may impact upon the workload of their subordinates.

The use of Lean as a large-scale quality improvement strategy has generated controversy on multiple fronts, but evidence on outcomes of Lean implementation is still emerging. Batalden et al. [49] have questioned the wisdom and appropriateness of implementing quality improvement systems such as Lean which are underpinned by “goods-dominated” logic systems in health care. Improvement strategies that recognize the dynamic environment of health care provision, where goods and services are consumed and produced simultaneously, may be more easily integrated into health care practice.

Normalization Process Theory offered a feasible and highly applicable model through which to assess implementation processes of the province-wide adoption of Lean. Use of a modified NoMAD survey allowed us to capture key elements of implementation that were of primary interest. The findings of this project provide a window into the perspectives of health care professionals who are in the midst of undergoing large scale transformation of a health care system and can inform future research, practice and policy related to implementation processes.

The implementation of Lean in Saskatchewan health care represents an ambitious, high-investment, multi-pronged attempt at a large-scale transformation “aimed at coordinated, system-wide change affecting multiple organizations and care providers” [4]. Recognizing that successful transformation is the result of constant small-scale changes to structures and process over long periods of time, Bohmer [35] suggests that the actual model of quality improvement (Lean or alternatives) is less important to achieving the goals of health reform than internalized repetitive and consistent process. The findings of our survey highlight that, while substantial progress has been made in the province-wide effort to implement, embed and integrate Lean into the Saskatchewan health care system over the past few years, ongoing support and innovative implementation strategies will be required if large-scale quality improvement initiatives are to become a routine part of practice for health care providers across the province and produce the intended improvements in quality and patient-centredness.