Effects of a national quality improvement program on ICUs in China: a controlled pre-post cohort study in 586 hospitals

In recent decades, critical care medicine has developed rapidly worldwide [1, 2]. The irreplaceable role of rescue treatment for critically ill patients has been widely accepted in intensive care units (ICUs). However, the severity of critically ill patients, a high intensity of therapeutic activities per day, a stressful environment, the quantity and complexity of medical data, and sudden changes in the patient’s condition cause ICUs to be associated with high risks of medical errors, hospital-acquired infections, and medical complications. Studies have reported that there is a high rate of medical errors and adverse event occurrence in ICUs [3]. Valentin et al. found that sentinel events related to medication, indwelling lines, airway clearance, and equipment failure in ICUs occur with considerable frequency [4]. Furthermore, an increasing number of evidence-based practices associated with acquired infection prevention and sepsis resuscitation might improve critically ill patient outcomes in the ICU, but eligible patients may not receive them. Hence, there is an urgent need for the development and implementation of new strategies with the aim of improving care quality in ICUs.

The effect of quality improvement (QI) programs on clinical practice and outcomes is gaining increasing attention in critical care medicine. Garrouste-Orgeas et al. reported that a multifaceted program was effective at preventing insulin errors and accidental tube/catheter removal [5]. Moreover, Cavalcanti et al. found that the implementation of a multifaceted QI intervention with daily checklists, goal setting, and clinician prompting did not reduce in-hospital mortality among critically ill patients treated in ICUs in Brazil [6]. A cluster randomized trial showed that multifaceted QI interventions improved the adoption of care practices in community ICUs [7]. Recently, McCredie et al. found that a trauma QI program improved clinical outcomes in severe traumatic brain injury patients [8].

However, the effect of a national QI program on ICU performance remains unknown in China. We sought to investigate the effects of a national QI program on ICUs in China. We assessed the association between this intervention and critical care quality performance and ICU mortality. Moreover, potential factors such as regional economic income and hospital/ICU structure resources were considered in the assessment of the effect of the interventions.

The study protocol was approved by the institutional review board of Peking Union Medical College Hospital; the approval included a waiver for the informed consent of the patients and physicians.

We used a pre-post cohort design to evaluate the implementation of a national QI program in ICUs in China. This program was led by the China National Critical Care Quality Control Center (China-NCCQC), which is the official national department that regulates ICU quality control in China. This program was considered an administrative strategy in these pilot hospitals.

We found that the implementation of a 3-year QI program period significantly decreased VAP incidence and improved DVT prophylaxis, SSC bundle compliance, and the microbiology detection rate before antibiotic use in Chinese ICUs. These findings support the hypothesis that the national QI program is effective at improving performance in the ICU setting.

Critical care medicine has made great progress, and a national quality control system for ICUs was established in China [12]. Professional qualifications for ICU attending doctors were implemented by the National Ministry of Health in China in 2009. Moreover, the specialist training project for critical care medicine started in 2019. Our present data show that ICUs have improved their attention to medical outcomes in the hospital during the past 3 years. In the present study, we focused on the effect of the QI program on three dimensions of ICU quality control indicators, including structure, processes, and outcomes, which are widely used to assess ICU performance [13, 14].

In the present study, the number of ICU beds maintained continuous growth from 2016 to 2018 in the 586 studied hospitals. Chinese society is becoming an aging society in the twenty-first century, and the number of hospitals and hospital volumes is gradually increasing. We think an aging society could also induce an increase in ICU admissions and ICU demand. However, the numbers of ICU doctors and nurses did not correspondingly increase with the increase in patients admitted to the ICU. Human resource indicators of quality control were not direct interventional targets in this study. Human resources may have been insufficient to keep pace with the increase in the number of ICU beds during the study period. Adequate human resource allocation is one of the core elements needed to ensure the quality of medical care [15]. An international investigation found that Asian ICUs had considerable variation in critical care structure, organization, and delivery, and single rooms and negative-pressure rooms were lacking [16]. Estenssoro et al. found that the availability of supporting specialists and key procedures was inadequate in Latin America [17]. Moreover, Sakr el found that a high nurse:patient ratio (> 1:1.5) was independently associated with a reduced risk of in-hospital mortality [18]. Recent literature on ICU structure in different regions/countries is summarized in Additional file 6. The annual ICU mortality rate and nurse-to-bed ratio in the present study were similar to those in US ICUs (ICU mortality, China vs USA 9% vs 10%; nurse-to-bed ratio, China vs USA 2:1 vs 1.7:1) [19]. Latin American ICUs had a lower nurse-to-patient ratio than Asian ICUs and a higher annual ICU mortality rate than US ICUs [16, 17]. Moreover, the bedside doctor-to-patient and nurse-to-patient ratios were not investigated in the present study. Here, using the doctor-to-bed ratio and nurse-to-bed ratio might roughly calculate the doctor-to-patient and nurse-to-patient ratios. In the present study, 1 doctor to 0.6 beds represented 1 doctor to 3–4 patients (equal staff 24/7). Moreover, 2 nurses to 1 bed represented 1 nurse to 2 patients (two shifts and absences for various reasons), which is similar to the results of the international investigation of Asian ICUs [16]. Here, we acknowledge that the lack of ICU human resources might impair the effectiveness of the QI program. The present study suggests that the relationship between medical human resources and ICU overexpansion was mismatched over the past 3 years. Increasing investments in ICU human resources could further improve the quality of ICU medical care in China.

For the quality control process of the QI program, we focused on sepsis bundle compliance, microbiology detection before antibiotics, and DVT prophylaxis. Our present study showed that QI program interventions could be effective at improving these process indicators. Previous studies have reported that QI programs/projects were effective at changing clinical practice [5‐7]. There is a high sepsis-related mortality rate in China [20]. The 3- and 6-h SSC bundles clearly outline the clinical path for the resuscitation of patients with sepsis and have been widely accepted and promoted over the past 10 years. Rhodes revealed a 40% reduction in the odds of sepsis patients dying in the hospital when the ICU was in compliance with the 3-h bundle [21]. Leisman et al. revealed that 3-h SSC bundle compliance was associated with improved survival and cost savings [22]. However, our study did not find that increased SSC bundle compliance decreased ICU mortality. We inferred that other non-sepsis-related mortality could confound the effect of SSC compliance on sepsis-related mortality. Further studies are required to elucidate the independent effect of sepsis compliance on mortality in septic patients in China.

For the outcome indicators of the QI program, we focused on hospital-acquired infection and ICU mortality. The hospital-acquired infection has become a major issue in ICU quality control. VAP, CRBSIs, and CAUTIs have always been considered quality control indicators for ICU performance, and serial bundles were validated to reduce the incidence of acquired infections in the ICU [23‐27]. In the present study, the QI program decreased the VAP incidence rate but not the CRBSI and CAUTI rates. Septic shock patients receiving mechanical ventilation therapy have a high risk of VAP. The proportion of admitted septic shock patients was 6.67% (2.82, 14.03)% in 2018. Here, we think that the improvements in sepsis bundle compliance and microbiological sampling before antibiotics might underlie the decreased VAP incidence rate.

Several potential explanations were considered for the lack of an effect on ICU mortality and other outcome indicators. First, the ICU human resource indicators were poor during the QI program period. Low doctor-to-bed and nurse-to-bed ratios were observed in 2018. Previous studies reported the effects of ICU structure indicators on ICU-acquired infections and clinical outcomes [3‐5]. Second, the follow-up period was long, and the clinical setting might have changed considerably during that time. Third, it is possible that the process care indicators have very modest or negligible effects on mortality. Although all the care process indicators included in the QI program in this study were recommended by ICU experts, most have uncertain effects on mortality. For example, DVT prophylaxis may decrease adverse events but have no proven effect on mortality [28]. A multicenter controlled pre-post trial found that a multicomponent intervention involving venous thromboembolism prophylaxis changed clinical practices but not clinical and economic outcomes [29]. Moreover, several clinical studies found that early goal-directed resuscitation did not decrease the mortality of septic shock patients [30‐32].

The subgroup analysis showed that the tertiary hospitals had better rates of microbiology detection before antibiotics and unplanned endotracheal extubation than secondary hospitals. Overall, in China, the medical quality of ICUs in tertiary hospitals is significantly better than that in secondary hospitals. This is related to the higher qualifications of medical staff, a higher number of patients, more financial support from the government or other sources, more standardized diagnoses and treatment processes and stricter quality management in tertiary hospitals than in secondary hospitals. In the present study, both tertiary and secondary hospitals were improved by the QI program. However, the VAP incidence rate significantly decreased in secondary hospitals. More attention should be paid to VAP in secondary hospitals. Moreover, the relationship between ICU volume and the outcomes of ICU patients is controversial. Kluge et al. found that ICU level was not associated with in-hospital and long-term mortality of ICU patients in the Netherlands [33]. Walkey et al. reported that academic hospitals in the USA with higher severe sepsis case volumes have lower severe sepsis hospital mortality without higher costs [34]. In the present study, tertiary hospitals had a higher ICU mortality rate than secondary hospitals. It is common that ICU patients with poor response to treatment will be transferred to tertiary hospitals from secondary hospitals in China. Hence, it is easy to understand a higher mortality rate in the higher ICU level. Further studies are needed to investigate the adjusted ICU mortality based on ICU level in China.

In this QI program, the impact of the regional GDP was considered in the evaluation of the effect of the QI program on ICU performance. A continuous decrease in the VAP incidence rate and a simultaneous continuous increase in the microbiology detection rate before antibiotic use were found in hospitals in high-income provinces but not in hospitals in low- and middle-income provinces. Bonell A [35] and Hurley JC [36] also observed low VAP incidence rates in high-income areas. This may be related to the financial advantages, human resource advantages and policy advantages of high-income areas in the field of health care. In the present study, the expansion of hospital and ICU volumes were obvious in low-income provinces over the 3 years of the QI program, but the corresponding ICU human resources were lower than those in high-income provinces. More ICU resources should be directed to low-income provinces to meet the increased demands on the ICUs.

Our study needs to be interpreted within the context of its strengths. We successfully assessed the effects of multifaceted QI interventions implemented as part of the national QI program led by the China-NCCQC. The trial was large, and the enrolled 586 hospitals represented 30/33 provinces of mainland China. Hence, the present study effectively reflected ICU performance at the national level. The intervention was effectively deployed by the administration of the China-NCCQC, and ICU quality control systems were established at the national, provincial and hospital levels. We used online administrative registries as our data sources and conducted a controlled pre-post trial as a practical means of evaluating the healthcare system interventions. This study design was efficient and allowed us to focus our resources on the interventions. Furthermore, we included a large number of hospitals in the study.

Several limitations should be acknowledged. First, some data on hospital mortality or 30-day outcomes, which could further indicate the effects of the interventions, were not collected in the present study. Second, compliance with some quality control processes was lacking. Third, there may have been confounding factors because of a lack of randomization and the absence of a control group. The outside factors, such as a broad range of sepsis bundles and guidelines and education on sepsis management, could drive secular trends in the improvement of sepsis care. In the present study, the objective conditions of the enrolled hospitals were comparable and stable, the interventions were clear, and the data quality was strictly controlled. Hence, this limitation might not impact the conclusion. Fourth, the corresponding outcome indicators of improvements in DVT prophylaxis and sepsis bundle compliance, such as the DVT incidence rate, pulmonary embolism incidence rate, and sepsis mortality rate, were not collected in this study. Moreover, subgroup information, such as surgical vs non-surgical patients and type of admission, was unavailable in the present study. Since our intervention consisted of a series of composite actions, it was difficult to evaluate the contribution of any single intervention in relation to the achieved results. Fifth, the same team was involved, leading to a risk of bias with regard to obtaining objective data. Moreover, the Hawthorne effect, which is a form of reaction whereby subjects improve or modify an aspect of their behavior when being studied, may also have existed in our study [5].

Among critically ill patients treated in ICUs in China, the implementation of a 3-year QI program period decreased the outcome indicator of VAP incidence and improved the process care indicators of DVT prophylaxis, SSC bundle compliance and microbiology detection rate before antibiotic use. However, the QI program did not decrease ICU mortality or other outcome indicators. Moreover, medical human resources might not have kept pace with the overexpansion of ICUs in China during the past 3 years.