The effectiveness of continuous quality improvement for developing professional practice and improving health care outcomes: a systematic review

Improving the quality and safety of health care is a priority of governments, health care workers, and the public [1, 2], with efforts often focused on investment in changes to the way health care is organized and delivered (system-level quality improvements) [3, 4]. While there are many different approaches that may be taken, continuous quality improvement (CQI) has received considerable attention within health care [5] as a way to enhance the quality of care and reduce costs [6‐9]. The use of CQI in health care has evolved since the 1990s, using quality control techniques and management theories employed in the industrial and manufacturing sectors [10‐14]. In its earliest form, CQI was based on five main principles, specifically: a focus on organizational process and systems, rather than on individuals within the system; the use of statistically and methodologically robust structured problem-solving approaches; the use of multi-disciplinary team working; empowerment of employees to help identify problems and action improvement opportunities; and, a focus on “customers” (i.e., public) through an emphasis on creating the best possible patient experience and outcomes [13, 15, 16]. As the use of CQI has grown in health care, and new approaches to quality improvement have emerged from industry (e.g., total quality management), it is evident that the core features shared by the different methods have evolved [17‐19]. A review of the characteristics of CQI in health care [20] identified three essential elements, which are systematic data-guided activities, iterative development and testing process, and designing with local conditions in mind [20]. Despite some uncertainty around the characteristics of CQI [21], several approaches encompass the fundamental principles and have been used in health, such as Lean Management, Six Sigma, Plan-Do-Study-Act (PDSA) cycles, and Root Cause Analysis [20].

Several systematic reviews have assessed the use of different approaches to help improve quality in health care, focusing on descriptions of the methods used and highlighting the differences in components included [22‐32]. Those assessing CQI were in specific populations or clinical settings, considering their application [29], effectiveness [31], and the barriers and facilitators to the implementation of CQI [28, 30]. None compared the effectiveness of CQI across a range of health settings, assessed the benefits of specific components, or considered the actual impact of the factors that may influence the effects of CQI. Given these limitations, we systematically reviewed the evidence to assess the effectiveness of different approaches to CQI for developing professional practice and improving health care outcomes in any health care setting. We aimed to examine the impact of the various components encompassed in, and that affect the application of, the different approaches, which may act as facilitators or barriers to change. These components were based upon previously identified common features within CQI [20, 33] and criteria used to evaluate quality improvement interventions [34]. Also, we intended to consider the influence of socio-economic health inequalities on the effectiveness, and the implementation, of the approaches to CQI in improving health care. The importance of socio-economic inequalities in determining health, and the use of health and social care services, is widely recognized [35]. Increasingly, efforts are focusing on incorporating consideration of health inequalities in developing health and social care services to address the widening health gap [36].

Our search strategy identified 7518 papers which, after duplicate removal, resulted in 6998 papers for inspection. Screening of titles and abstracts excluded 6718 records (Fig. 1). Manuscripts for 280 papers were screened, with 44 studies presented in 72 papers included in the review. Some 27 additional link papers were identified through checking study protocols and snowball sampling. Although 44 RCTs met the selection criteria, the results presented are for 28 RCTs comparing CQI with other non-CQI interventions, whether considered current usual practice (i.e., usual care, normal practice, delayed intervention, or waiting list (19 RCTs)), a new management intervention without a CQI component (7 RCTs) or where no description was provided of the comparator (2 RCTs).

Increasingly the provision of health and social care has been shaped by the challenges of a growing demand for services, pressures on available funding and a continued drive for efficiency [1]. Different approaches have been adopted in an attempt to maintain the comprehensiveness and quality of care, and to tackle inequity in provision of services [74]. Recently, attention has shifted to improving services by developing the capabilities and capacity of organizations through building their knowledge, skills, and infrastructure [74]. The focus on system-level quality improvement has resulted in CQI methods being identified, and increasingly used, as an approach to enhance the quality of care and reduce costs [3‐9, 72]. Despite its effectiveness within industrial and manufacturing sectors, it remains unclear whether CQI could be successfully employed in the health care sector. In systematically reviewing the evidence comparing the use of CQI with non-CQI interventions in health care, it was apparent that, regardless of the growth in evidence in the last 10 years, the results were largely equivocal. Although this appears to perpetuate much of the uncertainty, we identified elements of CQI that may prove beneficial in improving outcomes and possible reasons for our findings that may inform further research.

Our findings appear to concur with those of previous systematic reviews on developing professional practice and improving health care outcomes [26, 31], whether showing limited benefit [31], the influence of different components [26, 31], and/or reasons for the continuing uncertainties [22, 26, 28, 31, 32]. Where CQI appeared effective, collaboration and communication between health care professionals appeared important. We found that meetings helped to facilitate the implementation of CQI, particularly when meetings were led by participant leaders, who were an integral part of multidisciplinary teams, focusing implementation of initiatives through cooperative working. If these meetings were held frequently, such as weekly rather than monthly, this seemed to improve the effectiveness of the CQI approach taken. The importance of direct communication was re-enforced through the benefits reported for CQI initiatives that used person focused face to face training, which appeared more effective than other forms of training (e.g., web-based training or combinations of training methods) and were thought to help improve competence and motivation [75]. Others have found similar effects through different forms of interaction between those involved in CQI [28, 30, 76]. Audit and feedback have been recognized as important facilitators when implementing CQI, with increased intensity of support more effective in helping to incorporate improvements into practice [28, 30, 76]. The impact of collaboration and active communication may help to explain the apparent benefits from the use of CQI in primary care, where team structures reflect those used in operationalizing CQI methods [77, 78] and such initiatives are incentivized through other mechanisms (e.g., Quality and Outcomes Framework) [79]. Despite several different approaches to CQI, we identified that PDSA and MoI were the models most frequently used, showing benefit on clinical process and patient outcomes in a third of trials respectively. PDSA was previously reported to be an effective approach in improving health outcomes [32]. The rationale for the use of PDSA and MoI, and the reasons for their effectiveness in specific situations, has proven difficult to clarify. This may reflect the frequent adaptation of CQI models during implementation rendering the differences unclear [80], that models often have overlapping features [17] and frequently there is incomplete or inconsistent reporting of the details of the approach taken [22, 26, 32]. Although the evidence base has grown in recent years, there has been no discernible change in the effectiveness of CQI within the health care setting. This may be due to several factors; however, its likely to reflect the fact that studies undertaken are heterogeneous in nature through the approaches to CQI used, populations studied, and outcomes reported. Socio-economic health inequalities were not reported in any RCTs, which is not uncommon outside public health research, appearing to reflect their primary focus on the health condition and not the other underlying determinants of population health.

The limited effects of CQI initiatives may reflect several factors. First, health and social care organizations, both nationally and locally, are complex organizations which may lack the necessary structure, resources, and resolve to operationalize CQI initiatives effectively and consistently [81, 82]. Given the opportunity for approaches to CQI to be adapted to local conditions, there is a chance for variation in their implementation. This may reduce the inherent strengths of the CQI approach, limit its effectiveness and make it more difficult to research. Second, CQI initiatives are often implemented over a short period, restricting the opportunity to affect the different outcome measures assessed in the RCTs, particularly patient-based outcomes. Third, recognition of the importance of different components used in CQI (e.g., audit, feedback, meetings, and training), has resulted in their adoption as part of standard management practice. Consequently, they are increasingly part of different management interventions that are compared in trials, effectively controlling for their effects. Fourth, identifying the reasons underlying the effectiveness of specific approaches to CQI has proven difficult to clarify. This may reflect their frequent adaptation during implementation and that details of the approach were often incompletely reported [22, 26, 32]. Although a pragmatic approach to the use of CQI may be necessary in practice, adherence to the core components and more complete reporting of the different models used in trials would help to distinguish which models and elements are most effective [22, 26, 28, 31, 32]. Fifth, the limited evidence identified and its poor quality may result in uncertainty in the findings. The unclear or high risk of bias reported for most RCTs may reflect either the inherent challenges in conducting RCTs of CQI initiatives (e.g., blinding in cluster RCTs) or a lack of understanding of the importance of ensuring, and reporting, the rigor used in implementing the study methodology. It may be that the use of RCTs for evaluating CQI is undermined by the challenges faced and other approaches could compliment such experimental studies [82].

The systematic review had certain strengths, including the following: it was produced following a registered research protocol by independent researchers, clearly describing the methods followed; identified evidence through comprehensive searches of electronic databases, reference checking and citation checks; selected studies, extracted data, and assessed risk of bias using standard pre-piloted forms and processes; and involved public advisors in commenting on the research protocol and final report. Also, it had certain limitations, such as searches could have been extended to other sources; inclusion criteria were limited to RCTs which, although the gold standard for assessing effectiveness through limiting potential confounding, may restrict the opportunity to assess more real-world evidence provided by other comparative study designs; comparisons were with non-CQI approaches, removing the opportunity to directly compare between different CQI approaches; details of the studies were limited in the publications and further evidence was not obtained from study authors; extraction of data and assessment of risk of bias were undertaken by a single reviewer with information checked by a second reviewer, providing the opportunity for error; the synthesis categorized the evidence, limiting the extent of data presented from each RCT; and a meta-analysis was not undertaken.

Further research into the effectiveness of CQI interventions in health and social care would be beneficial. A systematic review comparing different CQI models and other active comparisons may help to identify the elements of these approaches that are useful to organizations. It could include experimental and non-experimental comparative studies and look at the specific influence of potentially important moderators (e.g., training methods/type and focus of meeting). If further RCTs are going to be undertaken it is important that they take a mixed-method approach, as it is currently unclear within the literature exactly which moderators are important. Any RCTs should be conducted by independent researchers that assess outcomes over a longer period, as this would help to clarify if the benefits could be realized in terms of clinical process or organizational outcomes and, more importantly, for patient-related outcomes. The RCTs could specifically compare the different key components that have been identified as core to the different approaches to CQI. Any RCT that is undertaken should report against a standard set of outcomes, provide full descriptions of all elements of the CQI process, and consider health inequalities. It has been evident that the quality of the evidence and the quality of its reporting is poor, preventing a full understanding of the findings and the context in which they have been attained. This should be addressed.

CQI is an important and proven approach to improving the quality and efficiency of industrial processes, which has drawn considerable and growing attention in health care. Evaluations of its use in health have been inadequate, causing uncertainty as to its benefits. It is evident that in certain situations, it has had significant effects on improving the provision of health care, although these were limited. Further independent research is required to clarify what approaches to CQI may be employed to improve the quality and efficiency of service provision.