http://orcid.org/0000-0001-7777-0909
Figures
Abstract
Background
Valid indicators are required to measure surgical quality. These ideally should be sensitive and selective while being easy to understand and adjust. We propose here the MTL30 quality indicator which takes into account 30-day mortality, transfer within 30 days, and a length of stay of 30 days as composite markers of an uneventful operative/postoperative course.
Methods
Patients documented in the StuDoQ|Colon and StuDoQ|Rectal carcinoma register of the German Society for General and Visceral Surgery (DGAV) were analyzed with regard to the effects of patient and tumor-related risk factors as well as postoperative complications on the MTL30.
Results
In univariate analysis, the MTL30 correlated significantly with patient and tumor-related risk factors such as ASA score (p<0.001), age (p<0.001), or UICC stage (p<0.001). There was a high sensitivity for the postoperative occurrence of complications such as re-operations (p<0.001) or subsequent bleeding (p<0.001), as well as a significant correlation with the CDC classification (p<0.001). In multivariate analysis, patient-related risk factors and postoperative complications significantly increased the odds ratio for a positive MTL30. A negative MTL30 showed a high specify for an uneventful operative and postoperative course.
Citation: Matthes N, Diers J, Schlegel N, Hankir M, Haubitz I, Germer C-T, et al. (2020) Validation of MTL30 as a quality indicator for colorectal surgery. PLoS ONE 15(8): e0238473. https://doi.org/10.1371/journal.pone.0238473
Editor: Ulrich Wellner, UKSH Campus Lübeck, GERMANY
Received: May 15, 2020; Accepted: August 16, 2020; Published: August 28, 2020
Copyright: © 2020 Matthes et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Due to data safety policy of the DGAV and german / European union data cannot be shared publicly. Data are available from the DGAV Institutional Data Access (contact via DGAV.de) for researchers who meet the criteria for access to confidential data. The authors of the present study had no special access privileges in accessing the data which other interested researchers would not have.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Background
Surgery is often the best and only therapeutic option available for a large number of diseases but comes with specific complication rates which increase with procedural complexity [1]. Treatment quality and the adequate management of complications arising from surgery are therefore critical factors for therapeutic success [2]. Numerous studies have shown that complication rates largely depend on patient-specific factors such as age or the ASA score on the one hand, and on the infrastructure of the treating hospital and its expertise for specific procedures on the other [3–7].
The quality of a given medical therapy can broadly be divided into 3 parts: The quality of the structure, which reflects the available equipment at a medical facility; the quality of the process, which reflects the way a medical treatment is carried out; and the quality of the result, which reflects the actual outcome of the treatment [8]. Quality indicators (QI) in surgery are measuring units which in general enable a graded distinction between good and bad operative/postoperative outcomes. They do not measure quality directly, but serve as surrogate parameters. A large number of parameters for visceral surgery have been described including 30- or 90-day mortality, postoperative length of stay, minor or major complications, anastomotic leakage, wound healing disorders, number of removed lymph nodes, quality of mesocolic excision, intraoperative blood loss, postoperative ventilation duration, resumption rate, and quality of life indices, which all could be used hypothetically to evaluate quality [9–11]. However, these indicators are influenced by a variety of factors, both patient-related (age, comorbidities) and the care system (availability of diagnostic and intervention options). They also require detailed and conscientious documentation. Ultimately, it is difficult to apply one of the QI listed above to a variety of operations in hospitals with different equipment.
An optimal QI should be applicable to all operations if possible, should be sensitive to the occurrence of complications, and should specifically depict comorbidities. In addition, it should be adjustable and take into account complex care structure across differently equipped hospitals.
We previously proposed the MTL30 as a general performance marker for surgical quality within the 30 day postoperative period [12]. In summary, if the patient died in inpatient care, MLT rated positive (M), whereas if they lived but are still in inpatient care at day 30, MLT rated positive (L). Further, if the patient was transferred to another acute care hospital (not for rehabilitation), MLT rated positive (T). Thus, the L and T designations reflect the occurrence of a major complication that either led to a significant extension of stay (L) or that could not be treated in the corresponding hospital (T). This allows adjustment between hospitals across different care levels. The time interval of 30 days was chosen for all designations to ensure comparability with the 30-day lethality. As it has been shown previously that daily documentation of complications is challenging, it is important to have access to pre-existing data [13]. The parameters that contribute to the MTL30 can be easily obtained from the hospital billing data which are universally documented and in contrast to the company's own documentations, are difficult or impossible to modify.
The German Society of General and Visceral Surgery (DGAV) have previously produced a large registry for patients to assess and to improve the quality of surgical treatment [14]. This includes the collection of data on the course of surgical treatment in hospitals in Germany. Importantly, all hospitals contribute to this registry, irrespective of their size or volume. This effectively reduces a potential selection bias, such as would occur if only high volume centers were included for instance, and enables the evaluation of quality markers over a wide range of patient groups and hospital categories.
Based on this registry, we conducted an analysis to characterize the MTL30 on colon and rectal cancer (StuDoQ|Colon Cancer; StuDoQ|Rectal Cancer registers) with respect to patient-specific factors and the sensitivity to complications.
Methods
The StuDoQ|Colon carcinoma and StuDoQ|Rectal carcinoma registers are prospectively documented databases for surgical interventions in colorectal carcinomas, which were set up by the DGAV in January 2010 (www.dgav.de/studoq, www.en.studoq.de). They were developed to facilitate the assessment of the quality and risk factors of colorectal cancer surgery in Germany. The declaration of consent, ethic approvment and the data security procedures were approved by the Society for Technology, Methods and Infrastructure for Networked Medical Research (http://www.tmf-ev.de). Written consent was obtained from all participants. The publication guidelines were determined by the DGAV (http://www.dgav.de/studoq/datenschutzkonzept-und-publikations guidelines.html). The data from participating centers are entered prospectively in pseudonymized form using a browser-based tool and subjected to automatic plausibility checks. Validation by cross-checking with institutional medical control data is part of the annual certification process. For the present study, all cases that underwent curative resection due to colorectal cancer were identified in the StuDoQ|Colon cancer and StuDoQ|Rectal cancer registry, and relevant demographic data, comorbidities, as well as information on operations, histology and perioperative history for analysis extracted in anonymous form. Full wall excision, simple polypectomy, endoscopic mucosal resection, and other endoluminal procedures, as well as palliative interventions regardless of the size of the operation, were excluded. Basic registration structures are comparable to the StuDoQ|Pancreas registration [14].
Postoperative complications included anastomotic leakage (grade C) [15, 16], infection of the surgical site [17], Clavien-Dindo classification (CDC) [18], burst abdomen, reoperation, and hospital mortality. They were defined as either present or not present. Additional postoperative parameters that were assessed were the need for unscheduled postoperative ventilation lasting more than 48 hours, pneumonia, length of stay (LOS), and readmission. Postoperative total morbidity was summarized as none (CDC 0), minor (CDC 1–2), severe (CDC 3a-4), and fatal (CDC 5) according to the CDC. Patients were counted as MTL30 positive if they had died within 30 days after index operation, the postoperative length of stay exceeded 30 days or if they had been transferred to another acute hospital or in hospital unit (e.g. transfer to a tertiary center due to surgical complications or to internal medicine due to postoperative pulmonary emboly). Transfer to a postoperative rehabilitation did not count as positive.
Statistical analysis was performed with a bilateral significance level of 0.05. Scale variables were expressed as median and range and categorical parameters as absolute frequency and percentage. Univariate analysis was performed using the chi-square test for categorical variables and the Mann-Whitney test for ratio variables.
All variables with a p-value <0.1 in the univariate analysis were included in the multivariate analysis. The multivariable analysis was carried out by logistic regression.
Results
A total of 19,646 patients diagnosed with colorectal cancer (66.6% colon resections, 33.4% rectal resections) were included in the evaluation. 55.85% (10,973) of the patients were male, the mean age at the time of surgery was 69.5 ± 11.9 years and mean BMI was 26 ± 4.96 kg/m2. Of all patients, 88.5% (17,930) were assigned to the ASA classification II / III. Tumor stages were UICC I 28.1% (4,963), UICC II 31.1% (5,485), UICC III 28.4% (5,000) and UICC IV12.4% (2,192), respectively. MTL30 positive were 9.59% (1,884) of all patients. For a detailed description of the patient population, see S1 Table.
In the univariate analysis, a positive MTL30 correlated with patient-specific pre-existing diseases. Accordingly, a positive MTL30 for patients with chronic obstructive pulmonary disease (8.94% vs. 20.19%; p <0.001), peripheral arterial occlusive disease (9.32% vs. 19.28; p <0.001), diabetes mellitus (NIDDM / IDDM) (8.72% vs. 11.48% / 16.12%; p <0.001), liver cirrhosis (9.28% vs. 23.1%; p <0.001), antihypertensive medications (6.1% vs. 11.95%; p <0.001), corticosteroids (9.48% vs. 18.52%; p <0.001) or alcohol abuse (9.28 vs. 17.31%; p <0.001) were significantly more frequently observed compared to the cohort in which these factors were absent. In the case of gradable risk factors (NYHA, cerebrovascular events) or the ASA score, the percentage of MTL30 positive patients correlated highly with the respective grade of the risk factor (Table 1).
There was also a significant correlation between positive MTL30 and tumor-specific parameters. The MTL30 was significantly more frequently positive in patients presenting with distant metastases, lymph node metastases and locally advanced tumors (T4) (Table 2). There was also a significant correlation with the pathological resection status (R0 9.34% vs. R1 15.02% / R2 22.77%, p <0.001) and tumor-related malnutrition (9.13% vs. 13.14%; p <0.001).
Focusing on a positive MTL30 as a function of postoperative complications, there was a significant correlation with both operative and nonoperative complications. The group of patients with postoperative burst abdomen, wound healing disorders, postoperative haemorrhaging, ileus, blood transfusion or re-operation each had a significantly higher percentage of positive MTL30 (Table 3). Similarly, patients with postoperative non-surgical complications, such as prolonged ventilation for more than 48 hours, renal failure, pneumonia, or thromboembolic events (pulmonary embolism, myocardial infarction, stroke) each showed a significantly increased percentage of positive MTL30. There was also a significant correlation between the percentage of positive MTL30 and the severity of the complication according to Clavien-Dindo (Tables 4).
The sensitivity that a positive MTL30 detects postoperative complication was increased with the grade of complication. While patients with a postoperative CDC 1 complication showed a positive MTL30 in 23.6% of all cases, this continuously increased to 94,1% of patients suffering from a CDC 4b complication (CDC 2: 26,8%; CDC 3a: 40,0%; CDC 3b: 47,5%; CDC 4a: 73,6%; CDC 5: 100,0%). On the other side, specificity for a negative MTL30 was 98.2% for patients without any complications, and slight decrease to 93.1% for CDC ≥ 4b. Also, specificity of a negative MTL30 was high to rule out occurrence of specific complication as AL (93,5%) or need for re-operation (95,1%).
The multivariable analysis for known patient-dependent factors as ASA or age as well as tumor-dependent factors as UICC-stage showed a significantly increased odds ratio (OR) for a positive MTL30. Similarly, the occurrence of postoperative complications increased the odds ratio for a positive MTL30. The 30-day lethality also showed a correlation with a number of risk factors and the occurrence of complications. But the 95% CI was smaller in all parameters arguing for a higher accuracy of the MTL30 (Table 5; Fig 1).
Discussion
Benchmarking is increasingly used within the healthcare sector as an approach to increase quality [19]. Nevertheless, valid assessment of surgical quality remains a major challenge [20].
Because morbidity and mortality decrease with the number of surgeries performed in hospitals [3, 4], “minimum quantities” have been recommended internationally. This has led to centralization in high volume centers [21, 22], but without necessarily improving surgical care. Indeed, it is the management of complications that appears to be better in high-volume hospitals. In addition, this volume-outcome relationship only applies to major interventions, since no volume-dependent differences have been shown for minor interventions [23]. A major problem with measuring the difference between surgical departments is that complications need to be documented in an unbiased manner, which necessitates an external audit. On the other hand, measured events need to be frequent enough to result in significant differences—particularly when cases are low. With the MTL30, we introduced a composite marker that allows a comparison between multiple hospitals across different levels of care and complexity of patient groups [12]. The MTL30 is based on the idea that there are no/minor complications if the patient can be released alive within 30 days of the operation. On the other hand, the operative/postoperative course must be regarded as conspicuous if the patient has died within 30 days, has been transferred to another acute care hospital, or is still in inpatient care after 30 days. Any serious medical intervention possesses a certain degree of morbidity and mortality, which explains why an MTL30 of 0% is extremely unlikely. The percentage of MTL30 positive patients is therefore dependent on the type of intervention performed (e.g. inguinal herniotomy vs. pancreatic resection), the patient's co-morbidity (e.g. ASA status), and the quality of the intervention. In addition, the index time-point of 30 days after operation was chosen for the MTL30 in accordance with the widely used 30-day mortality standard. Nevertheless, it can be adjusted to 7 or 14 days for other kinds of operations or other healthcare systems.
As a proof of concept, we showed for almost 20,000 datasets from the StuDoQ register that a positive MTL30 has a high correlation with existing risk factors of the patient, and strongly correlated with the occurrence of postoperative complications. As a composite indicator, the MTL30 is also more specific and sensitive than individual indicators.
Correctly determining surgical quality using QIs is a complex task [24]. On the one hand, markers should be used which are easy to collect, for example on the basis of accounting data, and which are difficult to falsify. This is indeed the case for the MTL30, since both the postoperative length of stay and the type of discharge from inpatient care (deceased, relocated, or discharged home/transferred to rehabilitation) are recorded in the accounting system. On the other hand, markers should be adjustable to ensure a fair comparison between hospitals. This was confirmed in the present study, where we used data from the StuDoQ register.
For a simple comparison between hospitals at different care levels, a global QI should ideally map all three pillars of quality to enable a fair comparison. If the outcome quality in terms of mortality is only considered, then the structural quality is overlooked. This includes e.g. the equipment of the hospital, the 24-hour preparedness for surgery, or the possibility of performing CT imaging. A low structural quality in a hospital therefore results in a high patient transfer rate, which is included in the recording of the "transfer" in the MTL30. In this way, it can be shown whether hospitals per se are appropriately equipped for the procedure performed. On the other hand, the length of stay (L) would be longer if hospitals were suitably equipped but are of worse quality due to disproportional complication rates.
Up to now, quality indicators are typically used to detect adverse events. By using the negative MTL30 as a quality indicator, we specifically tried to identify a “normal” postoperative course and could show a high degree of specificity (90%).
Determining significant differences between facilities depends on two basic parameters: The first is the number of cases carried out, and the second is the relative frequency of an "adverse" event. In order to show a significant difference between hospitals with, for example, 10% vs. 5% mortality, 185 interventions would be required [9] which increases to as many as 1, 000 cases in order to detect a statistically valid difference between 1% and 2% adverse events. In addition, mortality is a rare event overall, which explains why it is difficult to map a significant difference due to the small number of cases. Furthermore, mortality increasingly occurs after 30 days due to the continual improvement of intensive care medicine. For this reason, 90-day mortality is increasingly considered to be more appropriate [25]. However, these patients are still inpatient on the 30th day, and are therefore rated as conspicuous in the MTL30. Since the MTL30 subsumes several adverse events, and is therefore higher per se, the number of cases required to detect significant differences decreases. Furthermore, different operations can be clustered into groups of similar complexities to further increase the number of cases. In this way, a higher number of cases per hospital can be achieved, even with relatively rare interventions such as esophagus, stomach, and pancreatic resections.
One shortcoming of the MTL30 is that it does not detect minor changes within the postoperative course that do not prolong the in-patient care or lead to transfer to another unit (e.g. superficial side infection). Also, it does not differentiate between an eventful operative course from an eventful postoperative course. The major goal for further validation will be to determine the optimal cut-off timepoint for different operative procedures according to the nationally used healthcare system.
It has been shown previously that in addition to directly comparing between hospitals, self-evaluation leads to an increase in quality in individual hospitals, which is referred to as the Hawthorne effect [26]. The morbidity and mortality in participating hospitals decreased significantly due to the introduction of NSQIP [27]. As the MTL30 is easy to obtain and does not need extra documentation, it will give each individual hospital a quick and unbiased overview of where it stands in comparison with others.
In summary, measuring quality is a fundamental task in medicine. The MTL30 is an easily obtained and adjustable QI. A positive MTL30 shows high sensitivity for an eventful postoperative course, whereas a negative MTL30 has high specificity for an uneventful postoperative course.
Supporting information
S1 Table. Overview of patient characteristic.
https://doi.org/10.1371/journal.pone.0238473.s001
(DOCX)
References
- 1. Baum P, Diers J, Lichthardt S, Kastner C, Schlegel N, et al. (2019) Mortality and Complications Following Visceral Surgery. Dtsch Arztebl Int 116: 739–746. pmid:31774053
- 2. Ghaferi AA, Birkmeyer JD, Dimick JB (2009) Complications, failure to rescue, and mortality with major inpatient surgery in medicare patients. Ann Surg 250: 1029–1034. pmid:19953723
- 3. Diers J, Wagner J, Baum P, Lichthardt S, Kastner C, et al. (2019) Nationwide in-hospital mortality following colonic cancer resection according to hospital volume in Germany. BJS Open 3: 672–677. pmid:31592096
- 4. Diers J, Wagner J, Baum P, Lichthardt S, Kastner C, et al. (2020) Nationwide in‐hospital mortality rate following rectal resection for rectal cancer according to annual hospital volume in Germany. BJS Open.
- 5. Archampong D, Borowski D, Wille-Jorgensen P, Iversen LH (2012) Workload and surgeon's specialty for outcome after colorectal cancer surgery. Cochrane Database Syst Rev: CD005391. pmid:22419309
- 6. Nimptsch U, Haist T, Krautz C, Grutzmann R, Mansky T, et al. (2018) Hospital Volume, In-Hospital Mortality, and Failure to Rescue in Esophageal Surgery. Dtsch Arztebl Int 115: 793–800. pmid:30636674
- 7. Nimptsch U, Krautz C, Weber GF, Mansky T, Grutzmann R (2016) Nationwide In-hospital Mortality Following Pancreatic Surgery in Germany is Higher than Anticipated. Ann Surg 264: 1082–1090. pmid:26978570
- 8. Donabedian A (1992) The role of outcomes in quality assessment and assurance. QRB Qual Rev Bull 18: 356–360. pmid:1465293
- 9. Birkmeyer JD, Dimick JB, Birkmeyer NJ (2004) Measuring the quality of surgical care: structure, process, or outcomes? J Am Coll Surg 198: 626–632. pmid:15051016
- 10. Buhr HJ, Hardt J, Klinger C, Seyfried F, Wiegering A, et al. (2018) [Quality indicators with reference values and threshold limits in general and visceral surgery: For obesity and metabolic, pancreatic, colon carcinoma and rectal carcinoma surgery]. Chirurg 89: 1–3. pmid:29330678
- 11. Wiegering A, Buhr HJ, Klinger C, Furst A, Schiedeck T, et al. (2018) [Quality indicators for surgery of rectal cancer: Evidence-based development of a set of indicators for quality]. Chirurg 89: 26–31. pmid:29188353
- 12. Wiegering A, Wellner U, Seyfried F, Hardt J, Klinger C, et al. (2017) [MTL30 as surrogate parameter for quality of surgically treated diseases: Establishment based on the StuDoQ register of the German Society for General and Visceral Surgery]. Chirurg.
- 13. Dindo D, Hahnloser D, Clavien PA (2010) Quality assessment in surgery: riding a lame horse. Ann Surg 251: 766–771. pmid:20224375
- 14. Wellner UF, Klinger C, Lehmann K, Buhr H, Neugebauer E, et al. (2017) The pancreatic surgery registry (StuDoQ|Pancreas) of the German Society for General and Visceral Surgery (DGAV)—presentation and systematic quality evaluation. Trials 18: 163. pmid:28381291
- 15. van Rooijen SJ, Jongen AC, Wu ZQ, Ji JF, Slooter GD, et al. (2017) Definition of colorectal anastomotic leakage: A consensus survey among Dutch and Chinese colorectal surgeons. World J Gastroenterol 23: 6172–6180. pmid:28970733
- 16. Bruce J, Krukowski ZH, Al-Khairy G, Russell EM, Park KG (2001) Systematic review of the definition and measurement of anastomotic leak after gastrointestinal surgery. Br J Surg 88: 1157–1168. pmid:11531861
- 17. Kirchhoff P, Clavien PA, Hahnloser D (2010) Complications in colorectal surgery: risk factors and preventive strategies. Patient Saf Surg 4: 5. pmid:20338045
- 18. Dindo D, Demartines N, Clavien PA (2004) Classification of surgical complications: a new proposal with evaluation in a cohort of 6336 patients and results of a survey. Ann Surg 240: 205–213. pmid:15273542
- 19. Ettorchi-Tardy A, Levif M, Michel P (2012) Benchmarking: a method for continuous quality improvement in health. Healthc Policy 7: e101–119. pmid:23634166
- 20. Chung KC, Rohrich RJ (2009) Measuring quality of surgical care: is it attainable? Plast Reconstr Surg 123: 741–749. pmid:19182638
- 21. Whitehouse M (1995) A policy framework for commissioning cancer services. BMJ 310: 1425–1426. pmid:7613269
- 22. de Neree Tot Babberich MPM, Detering R, Dekker JWT, Elferink MA, Tollenaar R, et al. (2018) Achievements in colorectal cancer care during 8 years of auditing in The Netherlands. Eur J Surg Oncol 44: 1361–1370. pmid:29937415
- 23. Nimptsch U, Mansky T (2017) Hospital volume and mortality for 25 types of inpatient treatment in German hospitals: observational study using complete national data from 2009 to 2014. BMJ Open 7: e016184. pmid:28882913
- 24. Fink AS, Itani KM, Campbell DC, Jr. (2007) Assessing the quality of surgical care. Surg Clin North Am 87: 837–852, vi. pmid:17888783
- 25. Mise Y, Day RW, Vauthey JN, Brudvik KW, Schwarz L, et al. (2016) After Pancreatectomy, the "90 Days from Surgery" Definition Is Superior to the "30 Days from Discharge" Definition for Capture of Clinically Relevant Readmissions. J Gastrointest Surg 20: 77–84; discussion 84. pmid:26493976
- 26. Demetriou C, Hu L, Smith TO, Hing CB (2019) Hawthorne effect on surgical studies. ANZ J Surg 89: 1567–1576. pmid:31621178
- 27. Khuri SF, Daley J, Henderson WG (2002) The comparative assessment and improvement of quality of surgical care in the Department of Veterans Affairs. Arch Surg 137: 20–27. pmid:11772210