The evaluation of risk prediction models in predicting outcomes after bariatric surgery: a prospective observational cohort pilot study

Obesity is one of the twenty-first century’s pre-eminent public health problems. The World Health Organization (WHO) estimates that there are 2.3 billion overweight people globally, of which 700 million are obese (W.H.O 2018). A report by the UN Food and Agriculture Organization in 2013 showed that 24.9% of people in the United Kingdom (UK) were considered obese and that the UK was at the top of Europe’s obesity league table (The State of Food and Agriculture 2013). In the United States of America (USA), the prevalence is even higher with data showing that more than one in three adults are considered obese (Flegal et al. 2012).

High levels of obesity put significant burden on health services as a result of associated comorbidities. It has been estimated that the direct cost to the NHS of treating overweight and obese people was £4.2 billion in 2007 (Butland et al. 2007). The UK’s National Bariatric Register shows that 53.9% of men and 41.4% of women had four or more obesity-related diseases at the time of primary surgery (Welbourn et al. 2014). However, significant improvement, if not resolution, of comorbidities can occur within 2 years of bariatric surgery (Welbourn et al. 2014; Arterburn and Courcoulas 2014; Colquitt et al. 2014) with long-term cost savings due to treatment of not just obesity, but obesity-related illnesses ((UK) NCGC 2014).

The UK second National Bariatric Register report has shown that 16,956 primary bariatric surgical procedures were performed between 2001 and 2013, 95% of which were performed laparoscopically. In this cohort, surgical complication rates were 2.9% and observed in-hospital mortality 0.07% (Welbourn et al. 2014). With such low mortality rates, monitoring morbidity or complications may provide clinicians and patients with more useful information on quality and variation in standards of care and provide a greater opportunity for performance improvement.

Although weight loss is not considered to be the most important outcome of bariatric surgery (rather, the aim is to support resolution of obesity-related illnesses), it is nevertheless an important proxy of surgical effectiveness (Welbourn et al. 2014). Factors that have been found to influence various outcomes include higher body mass index (BMI), age, increase in number of comorbidities and American Society of Anesthesiologists’-Physical Status (ASA-PS) (Colquitt et al. 2014; Abraham et al. 2015). Of note, postoperative complications can vary in incidence depending on the definition of complication being used.

Finding an accurate risk stratification tool is important so that patients at higher risk of postoperative morbidity can be identified and their perioperative pathway optimised to drive better surgical outcomes. Studies have previously looked at the OS-MRS as a tool for prediction of perioperative outcome with variable results (Coblijn et al. 2016; Lorente et al. 2014), but this scoring system was designed and validated as a predictor of mortality and not morbidity (DeMaria et al. 2007). The Physiology and Operative Severity Score for the enUmeration of Morbidity and Mortality has been previously suggested as the most well-validated risk stratification model for predicting morbidity in heterogeneous patient populations (Moonesinghe et al. 2013), but previous research in bariatric surgery found it overestimated postoperative morbidity (Charalampakis et al. 2014).

The aim of this study was to evaluate two previously developed and validated scores, the POSSUM and OS-MRS scores, for the prediction of postoperative morbidity and longer-term weight loss at 1 year. In addition, we also evaluated independent predictors for poor weight loss using multivariable analysis.

This single-centre observational cohort pilot study was approved by the University College London Hospitals NHS Foundation Trust’s (UCLH) Research and Development office as a service evaluation. Between 01 March 2010 and 30 September 2013, data were collected prospectively on consecutive adult (> 18 years) patients undergoing bariatric surgery which included sleeve gastrectomy and laparoscopic Roux-en-Y gastric by-pass (RYGB) procedures at University College Hospital, a London teaching hospital.

In this study, we report the accuracy of POSSUM and OS-MRS in predicting postoperative morbidity in patients undergoing bariatric surgery. Both were shown to be poor predictors of POMS-defined morbidity on day 3 but showed moderate accuracy at predicting inpatient LOS ≥ 3 days due to morbidity or poor mobility. The average LOS of bariatric patients postoperatively has been reported nationally as 2.7 days (Welbourn et al. 2014), so these tools may be useful in predicting patients who are at risk of increased resource utilization. In this analysis, the strongest independent predictor of poor weight loss at 1 year (< 50% EBLW) was the failure to have > 28 days alive and out of hospital within the first 30 days of surgery—a composite endpoint of death, length of hospital stay and readmission to hospital.

It is important to risk stratify patients for bariatric surgery in order to facilitate optimal allocation of resources. There is no previously validated risk tool for sleeve gastrectomies and RYGB procedures, which comprise the majority of bariatric procedures. Three US studies have evaluated morbidity risk prediction models for bariatric surgery, all of which used the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) as their source of data. Turner et al. reviewed data from 2005 to 2008 and derived a normogram based on four factors (age, BMI, albumin and functional status); the C-statistic (which is equal to the AUROC) in the validation cohort was 0.629 (Turner et al. 2011). Although this study analysed over 32,000 bariatric procedures, these did not include sleeve gastrectomies (Gupta et al. 2011). Gupta et al. derived a risk prediction model based on six factors using the 2007 ACS-NSQIP dataset: the variables included recent myocardial infarction/angina, functional status, stroke, bleeding disorder, hypertension, BMI and type of bariatric surgery. The C-statistic for this model was 0.66 (Gupta et al. 2011) with almost a third of the cases were gastric band procedures, which are less commonly performed now. More recently, Aminian et al. used the 2012 ACS-NSQIP dataset to develop a model for laparoscopic sleeve resections comprising seven variables (congestive heart failure, chronic steroid use, male sex, diabetes, preoperative serum bilirubin, BMI and preoperative haematocrit). This model appeared to be the most promising, with a C-statistic of 0.682 (Aminian et al. 2015).

Our analyses found that POSSUM and OS-MRS were moderately accurate for predicting stay ≥ 3 days due to morbidity and poor mobility, with AUROC 0.63 for both. They were less accurate in predicting POMS-defined morbidity on day 3. Although they have not been shown to be a significant predictor of EBWL at 1 year, they do predict increased length of stay in hospital. Using one of these systems in the preoperative assessment clinic may support clinicians in identifying patients who may benefit from admission to the PACU (Daniel Guerron and Portenier 2016) or more intensive after-care pathways, including physiotherapy and occupational therapy. Preoperative optimization of these patients, or pre-habilitation, may be also of benefit to this demographic as it can improve physical fitness, which can help to improve outcomes (West et al. 2015; Bond et al. 2015).

Independent of the patient’s preoperative health status, a complicated postoperative course predicted poor weight loss at 1 year. This observation adds to the body of evidence that postoperative morbidity may have lasting impact on patient outcomes, which outlast the resolution of the overt complication, and which makes the prevention of postoperative morbidity an important goal of quality improvement (Moonesinghe et al. 2014; Khuri et al. 2005). Weight loss after bariatric surgery is not a certainty and requires the patient to be supported by a multidisciplinary team to achieve this. Patients require regular follow-up in the first 2 years post-surgery to ensure lifestyle changes occur, continued nutritional support and identify any maladaptive eating disorders (Metcalf et al. 2005). Postoperative surgical complications and prolonged recovery also have been shown to have an adverse effect on patient psychology (Pinto et al. 2016). Together, these factors may contribute to a poor outcome through lack of engagement and inability to access the necessary postoperative support as a result of their morbidity. In a patient population already at risk of depression (Carey et al. 2014), the added stress of complications may compound this risk and added to probable immobility as a result of postoperative complications, may result in a more sedentary lifestyle.

We also found an association between age and EBWL at 1 year, with a 3% increase in the risk of not achieving target weight loss, per year of advancing age. Previous analyses from large US cohorts have found conflicting evidence on this. A prospective observational study of 4776 patients evaluating 30-day outcomes (Flum et al. 2009) found no association between age and morbidity or mortality. A subsequent retrospective cohort analysis of 48,378 patients who underwent bariatric surgery in the 2005–2009 American College of Surgeons National Surgical Quality Improvement program (ACS-NSQIP) (Dorman et al. 2012) found older age was associated with prolonged LOS but not major adverse events. However, two more recent publications from the ACS-NSQIP of 44,408 (Khan et al. 2013) and 20,308 (Sanni et al. 2014) patients respectively showed an association between increasing age and morbidity and mortality. The latter study found that the odds of postoperative complications increased by 2% with each additional year of age. An analysis of 8945 patients from the Bariatric Outcome Longitudinal Database found that women and younger patients had significantly more weight loss (Van De Laar 2014).

Finally, comparing the POSSUM and OS-MRS, both show similar low accuracy in predicting postoperative morbidity at day 3 and moderate accuracy for predicting prolonged LOS due to morbidity or poor mobility, or ‘happy days’. As there is no tool for prediction of morbidity related to current bariatric surgery practices, either of these could act as a tool. This paper also highlights the need for a larger study to define a risk prediction tool for morbidity in bariatric surgery.

As the demand for surgery to treat the obesity epidemic increases, it will become increasingly important to risk stratify patients in order to effectively plan perioperative care. The mortality associated with surgery is very low but there is a need to reduce postoperative morbidity, which can have an effect on hospital resource utilization and is associated with reduced postoperative weight loss. Although the POSSUM and OS-MRS scores have been shown in this study only to be moderately effective at predicting outcome for both sleeve gastrectomies and RYGB procedures, they are equivalent to previously published analyses of other models in large US cohorts (Turner et al. 2011; Gupta et al. 2011; Aminian et al. 2015). Further, none of these US models have been validated on populations of patients undergoing the two most common bariatric procedures undertaken currently. Validation of our findings in multi-centre cohorts would be of value.