Predictive Value of POSSUM and ACPGBI Scoring in Mortality and Morbidity of Colorectal Resection: A Case–Control Study

One of the main concerns in POSSUM scoring is its overestimation of mortality. The mortality rate predicted by POSSUM (17.0%) was double the observed mortality in our total study population (8.9%). The drawbacks of the original POSSUM score led to the development of Portsmouth POSSUM and colorectal POSSUM. In our study, both scores underestimated the mortality risk (5.9% and 4.0%, respectively). Several reasons can be pointed out.

First, the primary studies on POSSUM extend their analyses back to the early 1990s and are less likely to represent current practice.13 Better understanding of diseases and improvements in diagnostic and therapeutic techniques have lowered mortality rates. Regarding surgical practice, developments such as laparoscopic intervention and enhanced recovery programs have caused a decrease in mortality.14 Hence, mathematical prediction models may be outdated. Law et al.15 reported overprediction of the POSSUM scores for laparoscopic colorectal resections. In converted controls, however, POSSUM scoring was reliable, which implies a discrepancy in predictive value due to operative technique. Second, POSSUM was originally developed with patients in the UK. However, outcomes may vary with other countries or high-volume specialized centers.16,17 Third, surgery got more and more specialized over time. The original POSSUM score was designed for the general surgical patient. The accuracy of these models is under discussion due to the use of mixed patient populations. More recently, several studies specify risk prediction for different subgroups.13

In our opinion, the main argument against the use of POSSUM in surgical audit is found in the validation as a risk prediction model. Nearly all reports on POSSUM scoring validate the score on their own series, which leads to different conclusions of reports regarding over- or underpredicting of the scores. Patient selection, local facilities, and skills may be confounding factors. This is illustrated by a broad range of observed vs. expected ratios in the literature (Table 5). In our opinion, risk prediction models need to be validated to a “gold standard” in order to allow comparative audit. Since reports on surgical outcome differ and definitions of adverse outcome may vary, this desired standard may be unrealistic. Russell18 and Ferjani11 have stated that a system with standard definitions is mandatory before clinical performance can be compared between health care systems and institutions. A proper and uniform definition of mortality is essential in risk prediction. Most studies on POSSUM describe mortality as primary outcome only. As Brooks et al.6 pointed out, the majority of surgical procedures carry a low risk of death. However, along with decreasing mortality rates, the relevance of predicting morbidity is increasing. POSSUM also predicts the chance that a patient develops one or more complications with only moderate accuracy (area under the curve 0.53–0.76).

Cumulative sum techniques (CUSUM), described in 1954 by Page and its first introduction in surgical practice in 1994, might encounter the drawbacks mentioned above. This technique allows one to judge whether an observed variation in performance is acceptable (i.e., probably due to chance) or whether the variation is greater than what could be expected from random variation and thus may be a cause for concern. However, acceptable and unacceptable outcome rate as type I and II error rate has to be defined first. CUSUM is helpful in the evaluation of a clinical procedure before its implementation without the drawbacks of a randomized clinical trial. Plotting of the cumulative sum has been proven valuable for examining sequential measures, detecting changes over time, and is applied as a means of assessing surgical skills of trainees. Continued surveillance using the CUSUM allows the early detection of factors that lead to an increased failure rate. Quality control and objective and quantified recording of the findings meet the recommended criteria for medical audit.19‐23

By tailoring POSSUM to patient- and procedure-specific assessment, it becomes a tool that can help inform the individual patient on a certain procedure and the risk on adverse outcome. Several studies reported the value of POSSUM in surgery for colorectal cancer.9,17,24‐27 Tekkis et al.4 developed the colorectal POSSUM and differentiated for elective or acute procedures and procedures performed for malignancy or no malignancy. Constantinides et al.28 studied the value of POSSUM scoring in patients with complicated diverticulitis and concluded that CR-POSSUM was more accurate to predict outcome than (P-)POSSUM. Oomen et al.29 retrospectively compared the different POSSUM scores in 241 patients undergoing elective resection of the sigmoid for carcinoma or diverticular disease. Although patients with diverticular disease had a higher score than patients with malignancy, mortality rate did not differ. It was concluded that none of the POSSUM scores was predictive of disease-specific mortality. However, we found significant differences in POSSUM scoring related to the indication of surgery. All POSSUM scores predicted outcome more accurately in patients with diverticular disease than in patients operated on colorectal cancer. Within the subgroup of patients with diverticular disease, we could not define a superior POSSUM score. POSSUM, P-POSSUM, and CR-POSSUM scores also predicted equally in patients with colorectal cancer. In our opinion, disease-specific patient and operative variables should be included to improve the scores. Furthermore, patients are getting older and preexistent morbidity is likely to increase. In our series, a larger variation of the various POSSUM scores is found in octogenarians (Fig. 4). This is in accordance with Slim et al.6 who studied risk prediction by POSSUM and the AFC index (Association Française de Chirurgie). It is unclear whether the introduction of more extensive cardiac and pulmonary risk indexes might further improve the predictive accuracy of POSSUM scoring. It may further complicate POSSUM scoring. The AFC index is a simpler instrument without any mathematical formulas. It uses only four independent preoperative factors and is found to be as predictive as P-POSSUM.

All mean POSSUM scores were higher in the carcinoma group than in diverticulitis, whilst observed mortality rates were comparable.

ACPGBI scoring was found to be superior in predicting mortality after resection of colorectal cancer both in elective and acute interventions, which is consistent with the literature.11,30 Thirty-seven out of 386 patients operated for colorectal malignancy (9.6%) had known metastasis. Mortality rate was 29.7% (11/37); all patients died as consequence of a change to tender loving care due to a lack of perspective on a reasonable outcome (Table 3). Mean POSSUM, P-POSSUM, and CR-POSSUM score in the deceased group were lower than in patients who survived (respectively 11.7 vs. 21.1, P = 0.02; 3.9 vs. 7.8, P = 0.05; and 2.5 vs. 3.6, P = 0.07). Although based on a small population, these results demonstrate the insufficient predictive value of POSSUM scoring in patients with extensive oncological disease. Patients with colorectal cancer are likely to be immunosuppressed due to elderly age, nutritional status, and the colorectal cancer itself.31,32 The Dukes’ classification is too coarse to reflect today’s pathologists’ power to detect disease parameters in cancer.29 Implementation of nutritional status in POSSUM might help improve the area under the curve in malignancy.4,7,26 Both suggestions for improving POSSUM scoring need further research.

Question remains whether or not these patients have to be taken into account in validating risk prediction models. Well-informed patients with advanced cancers may trade off a short-term risk in exchange for cancer cure. In this population, the risks of resectional surgery may outweigh the benefits of a simpler and possibly safer palliative operation, but this requires reliable risk estimations.12

No previous studies evaluated POSSUM scoring in patients with inflammatory bowel disease. Due to the view of the physiological variables included in POSSUM, the younger, relatively healthy patient with inflammatory bowel disease is likely to have a different score than the elderly with an extensive medical history operated for colorectal carcinoma. Patients with colitis often have an increased white blood cell count and low levels of hemoglobin or albumin, reflecting disease activity. Furthermore, these patients often use immunosuppressive medication and have a poor nutritional status, which is found to increase adverse outcome after surgery.33 We found lowest POSSUM scores in this subgroup for mortality, which corresponded with the observed death rate. However, morbidity was underestimated by POSSUM. Younger age and the absence of cardiopulmonary comorbidity may explain the capability to overcome postoperative complications. POSSUM scoring for IBD may require a more prominent role of age, use of medication, nutritional status, level of hemoglobin, albumin, and white blood cell count.26 Calibration of POSSUM for patients with inflammatory bowel disease may be hard since recent review showed improved outcome of surgery to be highly dependent on accurate timing of the surgery and better perioperative care.34

The most reliable predictions as demonstrated by the highest areas under the curve were found for patients with diverticulitis. The observed mortality was considerably higher than in patients operated for inflammatory bowel diseases and almost similar to patients with carcinomas. Patients with diverticulitis had the highest body mass indexes and were operated urgently more often, both associated with an increased complication rate.35‐37 Left-sided resections were more frequently performed in patients with diverticular disease (81.3%) than in malignancy (50.3%) and inflammatory bowel diseases (15%). Left-sided resections are known to cause more complications.33 Another explanation may be patient selection. High-risk patients with diverticular disease may be withheld from surgery, whereas a malignant indication for surgery will not allow a conservative treatment strategy. Accurate definition of high-risk patients is essential. Body mass index, operative urgency, and degree of peritoneal contamination may be important variables in order to calibrate POSSUM scoring for diverticulitis.

This study questions the role of POSSUM for the comparison of clinical performance between health care institutes. Poor definitions of surgical outcome and problematic validation of this risk prediction model are the main objections to use POSSUM for surgical audit.

In its present form, POSSUM scoring should not be used for medical decision making in individual patients either.

Future investigation needs to point out whether further calibration of POSSUM is feasible, or that alternative risk prediction models need to be developed. One solution may be for models to be more disease-specific.