The association between quality system development stage and the implementation of process-level patient safety themes in Dutch hospitals: an observational study

Growing concern about quality and safety within healthcare organizations made quality improvement an important topic in healthcare. According to Donabedian’s well-known model of quality improvement, quality can be achieved through a continuous cycle in which the organizational structure enhances organizational processes, which in turn leads to improved outcomes at the patient level [1‐3]. In this model, the organizational structure is the quality system and is a prerequisite for continuous improvement [1‐3]. A quality system is defined as a set of interacting activities, methods, and procedures aimed at directing, controlling, and improving the quality of care [4]. Quality systems contain quality improvement strategies in different areas such as policy, healthcare staff, patient involvement, and systematic measurement of outcomes [4‐8]. The quality system is the structure within which quality improvement policies are embedded and this is hypothesized to have an influence on quality improvement activities at the process level. Ideally, in a developed quality system the quality activities are integrated into daily working processes throughout the healthcare organization. The policies and management of the healthcare organization ensure that this is done at the process level of organizations as well and this should become visible in a reduction of variation in processes of the healthcare organization and improvement of these processes over time. Most healthcare organizations have now implemented quality systems. However, there is large variation between countries and between individual healthcare organizations in terms of how well developed their quality system is [5, 7, 9, 10]. Insight into the development stage of a quality system is important, because it is assumed that this shapes the organizational processes, in turn leading to higher quality of care at the patient level [9, 11].

Several studies examined the relationship between quality systems or derivatives (such as hospital-level accreditation) and outcomes in terms of quality of care. Shaw et al. studied the effectiveness of different forms of external quality assessment of hospitals and found that accredited hospitals performed better on patient safety outcomes [11, 12]. Weiner et al. linked quality improvement with a set of patient safety indicators at the organizational level and found that higher percentages of physicians participating in quality improvement teams led to fewer postoperative complications and lower rates of technical difficulties with procedures [13, 14]. Kunkel et al. found higher scores for structure and outcomes when the implementation of a quality system was initiated by managers and when staff provided input to the quality system design. Subsequently, this was found to result in more advanced quality systems [15, 16]. Groene et al. found that better-developed quality systems were associated with lower rates of hospital complications and to some extent with fewer hospital readmissions in Spanish hospitals [9]. However, the same study found no association between the maturity of the quality system and hospital mortality and length of admission [9]. The European research project DUQUE assessed the association between quality management and patient outcomes in a wider setting: the European Union [17, 18]. Results from this project showed some associations between quality management measures at the hospital level and quality measures at the department level [19]. However, these associations were weak and the variability between countries was high [19].

Despite these examples from the literature, research into the relationship between quality systems and measures of quality and safety is limited and often restricted by small sample sizes and lack of availability of sufficient outcome measures [13, 14]. Although implementing quality systems in healthcare aims to improve the quality of care and patient safety by improving the processes, no clear evidence can be found in the literature that this is actually the case. This study aims to provide more insights into the association between the development stage of a hospital quality system (structure) and the quality of care at the process level. It should thereby be able to shed light on how quality systems work and provide more insights into the relationship between structure and processes in quality improvement. In this study, processes are quantified by the degree of implementation of patient safety themes within a national patient safety program. The degree of implementation of the patient safety themes is reflected in the scores for process indicators. The program will be described in more detail in the methods section. The research question addressed in this study is: Is there a positive association between hospital quality system development stage and the implementation of patient safety themes on process level? In line with Donabedian’s principles of quality improvement outlined above, we expect a positive association between the development stage of hospital quality systems and the implementation of patient safety themes. In other words, the patient safety themes measured in this study are expected to be more thoroughly implemented in hospitals with better-developed quality systems and this should be reflected in higher scores for the process indicators.

In total, data from 30 hospitals and 12,485 observations were included in this study. Table 2 shows the results of the quality system development stage variables and the results of the process indicators of the patient safety themes. The mean score for hospital quality system development stage was 2.30 (range 1 to 3). This indicates that all hospitals have implemented a quality system into their organization, and that most hospitals have designed quality improvement projects but don’t systematically use performance measures to adjust quality policy. The mean scores for the development stage for the different quality system domains ranged from 1.56 for Patient Involvement to 2.66 for Human Resources Management (range 1 to 3). This indicates that most hospitals have developed activities to train and educate healthcare staff in quality methods, but to a lesser extent involve patients in quality activities. The mean percentages for process indicators for the patient safety themes ranged from 12% for Pain indicator (standardized pain measurements, three times a day in the first three days after surgery) and Medication Reconciliation at Discharge to 73% for Time Out Procedure Execution (TOP). This indicates large differences between hospitals in the degree to which they have implemented the patient safety themes. Furthermore, the combination of the high SD and the broad range in scores reveal that most of the scores lie in both sides of the extremes. Analyses showed that this is being caused by differences between hospitals.

Table 3 shows the variation between hospitals in the association between hospital quality system development stage and scores for process indicators for the patient safety themes. The wide range in scores for the various process indicators indicates large differences between hospitals in the implementation of the patient safety themes. For example, there was one hospital in our sample performing Medication Reconciliation on Admission only in 0.6% of admissions, whereas another hospital performed Medication Reconciliation in 96% of admissions. And for Contrast- Induced Nephropathy, the percentage of high-risk patients who were hydrated before undergoing contrast administration ranged between hospitals from 22% to 93%. The ICCs of the intercept-only model ranged between 23.1 for Contrast- Induced Nephropathy and 58.1 for Medication Reconciliation on Admission. The ICC indicates the percentage of the total variance in scores for the process indicators that came from the hospital level. For example, an ICC of 58.1 means that 58% of the total variance was related to differences between hospitals. The ICCs found in this study indicate that the differences between hospitals were relatively large in our sample. The ICC decreased when the hospital quality system development stage variable was added to the model, this decrease was statistically significant. In this model, the ICC ranged from 21.1 for Contrast-Induced Nephropathy to 58.1 for Medication Reconciliation on Admission. The decrease in ICC indicates that the differences between individual hospitals on the process indicator scores can be partly explained by differences in the development stage of the hospitals’ quality systems. The remaining variance can be explained by other differences between hospitals.

Table 4 shows the association between hospital quality system development stage and scores for the process indicators for the patient safety themes. For four of the five safety themes, positive associations were found between the overall quality system development stage and the scores for the process indicators. However, none of these associations were statistically significant. Inconsistent results were found when examining the associations between the different dimensions of the quality system development stage. Some of the associations were positive and some were negative, but only a few were statistically significant. Statistically significant positive associations were found between the Policy and Strategy dimension and the patient safety themes Medication Reconciliation at Discharge and Contrast- Induced Nephropathy. Statistically significant positive associations were found between Patient Involvement and the patient safety themes Contrast- Induced Renal Failure and High-Risk Medication. One additional analysis was performed, as the association between the hospital quality system development stage and the process indicators for the patient safety theme Wrong Surgery was in the opposite direction to what had been hypothesized. This association was investigated in more detail by examining the association between the hospital quality system development stage and the three individual checks of the TOP. Negative associations were found between the development stage of the quality system and the checks on the patient identity and the check on the side/site.

Quality systems are hypothesized to have an influence on quality improvement activities at the process level, which in turn influences hospital outcomes at the patient level. This study linked the development stage of a hospital quality system (structure) to the implementation of patient safety themes at the process level (processes) in Dutch hospitals. This was measured by means of process indicators. We found no statistically significant associations between the development stage of a quality system and the implementation of patient safety themes. Some statistically significant associations were found between dimensions of the quality system development stage and process indicators for the implementation of patient safety themes. However, given the large number of associations that were tested compared to the limited number of statistically significant associations found, it is possible that these findings can be attributed to chance. We therefore conclude that this study found no conclusive evidence for a positive association between the development stage of a hospital quality system and the implementation of patient safety themes at the process level. An interesting finding is the large variation in scores of hospitals on the implementation of the patient safety themes.

The results of this study could indicate that no association exists between structure and processes in the cycle of quality improvement. However, other explanations seem more likely. These might be associated with the translation of the quality systems into quality improvement activities at the process level on the one hand, or with the complexity of implementation itself on the other. Firstly, several factors could hinder the translation of structural factors into more practical activities that can be applied on the work floor. For example, the extent to which hospital management uses a top-down approach to make a connection between the quality system and processes in their hospital could influence the extent to which quality system components are adequately translated into quality activities at the hospital department level. Alternatively, the attitudes of healthcare staff to quality improvement might play a role in the extent to which parts of the quality system are adopted and adhered to at the clinical level These factors might mediate the relationship between quality systems and processes, making it complex to measure and attribute to the variation of scores on process indicators that we found in this study. Future research into the attitudes of healthcare staff towards quality procedures and the extent to which these are applicable and adhered to in daily practice could provide insights in the context in which an association between a quality system and process indicators is being established.

Secondly, the structure of quality management is unlikely to be the only explanatory factor in the degree of implementation of patient safety themes. Implementation is seen as highly complex, as the extensive body of literature on barriers and facilitators for implementation illustrates [22‐25]. Besides organizational aspects that can facilitate implementation of interventions into healthcare practice, individual factors play an important part as well. More specifically, implementation is assumed to require behavioral changes by individual healthcare staff and numerous theories on behavioral change have been developed to guide interventions. Recently, the theoretical domains framework has captured the key concepts of all these different theories in a comprehensive framework of 14 domains that can help to explain implementation problems and provide input for the design of interventions [26, 27]. The theoretical domains framework highlights not only the importance of a theoretical grounding in the design of interventions but also the broad spectrum of concepts that are associated with healthcare staff behavior [26, 27]. These behavioral components were not measured in the present study but might have been important in the level of implementation of the patient safety themes and might explain the large differences that between hospitals.

Lastly, it can be argued that the indicators used in this study cannot be seen as a representative set of indicators to define an association between the quality system development and process indicators since there is no direct match between the indicators of the quality system and the process measures. However, according to the idea of quality improvement a quality system sets the preconditions for the organization to improve quality by integrating quality improvement policies in daily working processes. This would still lead to the expectation that the quality system has a general effect on outcomes at process level even if there is no direct link between the indicators. Related to this issue, the process indicators that were used in this study were part of a national safety programme that was designed separately from the existing quality system of the hospitals. Therefore these indicators are possibly less integrated in the hospital organization, making it difficult to find an association between the hospital quality system and the chosen process indicators. In future research, process indicators with a more direct link to hospital quality systems could be considered.

This study found no association between the development stage of a hospital quality system and the degree of implementation of patient level safety themes at the process level. This rejects the hypothesis that quality improvement is caused by a positive association between structure and processes, which in turn contribute to outcomes at the patient level. Several factors may have contributed to the results of this study. Future research should try to resolve methodological constraints associated with the measurement of quality systems as well as quantifying more factors associated with the implementation of quality improvement interventions.