Clinical prediction score for superficial surgical site infection after appendectomy in adults with complicated appendicitis

This study was a part of a randomized controlled trial (RCT) comparing SSI between delayed and primary wound closure in complicated appendicitis [12]. Patients were recruited from two university hospitals (i.e., Ramathibodi and Thammasat University Hospitals) and four regional hospitals (i.e., Lampang Hospital from the North, Chonburi hospital from the Eastern, Surin form the North-eastern, and Pathumthani hospital from the Central). Preoperative inclusion criteria were adult patients, aged ≥ 18 years, who had appendectomy via right lower quadrant abdominal incision. The intraoperative eligible criteria were complicated appendicitis (i.e., gangrenous and rupture). Definition of gangrenous appendicitis was inflamed appendix with necrotic wall (dark, grayish color); ruptured appendicitis was defined as presence of a hole in the appendix, intraoperative rupture, or presence of frank pus. Exclusion criteria were obesity (body mass index (BMI) ≥ 40 kg/m²), autoimmune disease, end-stage renal/liver disease, or human immunodeficiency virus. Preoperative and postoperative intravenous antibiotics covering gram-negative and facultative/anaerobic bacilli had been regularly prescribed in all patients until their body temperature (BT) < 37.8 °C for 24 to 48 h, then switched to oral antibiotics for 7 to 10 days as specified in the RCT protocol [12]. The study was reported according to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement [7]. The original study was approved by the Ethics committee of Ramathibodi Hospital and the collaborating hospitals, in addition to Chonburi hospital for the current study.

SSI was defined following the Center of Disease Control (CDC) criteria as infection within 30 days that involved skin and subcutaneous tissue, with any of the following: purulent drainage, positive culture of organism isolated from fluid or tissue, or any of the following symptoms/signs: pain or tenderness, localized swelling, redness/heat, or the wound was opened by physician without a positive culture [13]. SSI was assessed before discharging home, at 1-week and 1-month follow-up.

Predictors included in the analysis were patient-related and operative-related factors. Patient-related factors were age, gender, BMI, active smoking, American Society of Anesthesiologists’ (ASA) classification, diabetes, hypertension, duration of symptom before admission, presence of fever at admission (BT ≥ 37.8 °C), white blood cell count, and anemia (hemoglobin ≤ 10 g/dL). Operative-related factors included wound length, subcutaneous fat thickness, type of appendicitis (gangrenous or ruptured), wound contamination with exudative fluid, frank pus, fecal contamination (presence of fecolith or wound contamination with fecal material), presence of phlegmon, operative time, and use of closed suction drain.

As for a rule of thumb of simulation study, at least 10–20 of interested events (i.e., SSI) were required per one predictor in the final model to prevent model optimism [14]. Our data contained 48 SSIs [12]; thus, the final model should include not more than five predictors.

Baseline characteristics of the patients were described. Predictors were compared between SSI and non-SSI using chi-square (or exact test) and Student’s t test (or Mann-Whitney U test) for categorical and continuous data, respectively. Variables with p value less than 0.10 would be included in multivariable analysis. For adjusting purpose, type of wound closure (i.e., delayed primary or primary closure) was included in the multivariable analysis. Forward stepwise logistic regression was applied to identify predictors that significantly associated with SSI (p value < 0.05) and thus should be kept in a final parsimonious model.

The coefficients of significant predictors in the parsimonious model were then used to generate risk scores. Then, area under the curve (or concordance (C) statistics) along with its 95% confidence interval (CI) was estimated to quantify discrimination performance. Calibration of the model was assessed using Hosmer-Lemeshow goodness of fit test. Sensitivity, specificity, and positive likelihood ratio (LR) were also estimated. Cutoff point for risk stratification was calibrated according to the LR performance. Posttest probability was finally estimated according to the stratified risk groups.

An internal validation of the risk prediction score was assessed using a bootstrap with 1000 replications [15]. For calibration, the Somer’D coefficient which is an estimate of the correlation between the observed and predicted values of SSI was estimated for all bootstrap data (called D_boot) and derived (original) data (called D_org). A bias was then estimated by subtracting the D_org with the mean D_boot, lower value reflected less bias and thus better calibration. In addition, the original C statistic was compared to an average C statistic from the bootstraps for discrimination performance. All analysis was performed using STATA version 15.0.

Univariable analysis was performed to assess association between each risk factor and SSI, see Table 1. For ease of application, continuous variables were categorized into two groups based on 75 percentiles except operative time that was categorized according to National Nosocomial Infections Surveillance (NNIS) [16]. The cut points were 25.7 kg/m², 48 h, 7 cm, 4.5 cm, and 75 min for BMI, duration symptoms, incisional length, subcutaneous fat thickness, and operative time, respectively. Among 19 predictors, eight had p <  0.10 including BMI, diabetes, duration of symptoms, appendicitis classification, incisional length, subcutaneous fat thickness, fecal contamination, and length of operative time. Stepwise logistic regression identified four predictors in the parsimonious model, i.e., presence of diabetes, incisional length, fecal contamination, and operative time > 75 min, see Table 2. The risk prediction equation was written as

The C statistics of this model was 0.74 (95% CI 0.66, 0.81) indicating acceptable discrimination of SSI from non-SSI. Hosmer-Lemeshow goodness of fit test was applied and demonstrated good calibration of the model (chi-square = 4.42, p value = 0.491), see Additional File 1: Table S1. Coefficients of significant predictors were used to calculate prediction score. Diabetes and incision length were weighted 1; operative time and fecal contamination were weighted 1.2 and 1.3, respectively. Adding them up resulted in risk score ranged from 0 to 4.5, see Table 2. Cutoff points were identified based on positive LR of each distinct score from ROC curve analysis. The score was classified as 0, 1–1.3, 2–2.5, and > 2.5 for very low, low, moderate, and high risk groups, respectively. Sensitivity, specificity, positive LR, and posttest probabilities were demonstrated in Table 3. Fagan’s nomogram was constructed, and posttest probabilities were calculated based on incidence of SSI in this study which was 8.7% (95% CI 6.4, 11.0), see Fig. 1 [12]. Patients had probabilities of SSI of 12.2 (95% CI 9.0, 15.5), 24.0 (95% CI 18.5, 29.5), and 46.9 (95% CI 38.8, 53.9) for low, moderate, and high risk groups, respectively.

A bootstrap with 1000 replications was performed to assess internal validation, which yielded estimated D_org and D_boot coefficients of 0.476 and 0.500 (95% CI 0.496, 0.506) for the derivative and bootstrap models, respectively. The average bias was only − 0.025 (95% CI − 0.030, − 0.019), suggesting good calibration. The bootstrap C statistics was 0.750 (95% CI 0.748, 0.753), with a bias of − 0.012 (95% CI − 0.015, − 0.010).

Applying our model is simplified by counting number of four risk factors (i.e., diabetes, operative time > 75 min, fecal contamination, and incision length > 7 cm). According to the risk stratification scores, patient with 0, 1, 2, and > 2 risk factors will be respectively classified into very low, low, moderate, and high risk groups with positive LR for low, moderate, and high scores of 1.4, 3.3, and 9.2, in which positive LR of 5 to 10 could moderately shift pre- to posttest probability as for EBM working group recommendation [26]. To calculate posttest probabilities of SSI, Fagan’s nomogram could be used or to estimate posttest probability of SSI occurrence according to risk prediction score [27]. For instance, if patient has history of diabetes, operative time > 75 mins, and fecal contamination, she/he will be classified as high risk group (three factors); posttest probability of SSI for this patient is about 46%. If patient does not have history of diabetes, but has fecal contamination and incision length > 7 cm, this patient will be classified as moderate risk and thus her/his posttest probability is about 20–30%.

The results from previous randomized controlled trial [12] demonstrated that delayed primary wound closure had no benefit over primary wound closure in normal risk patients and primary wound closure should be done with risk of SSI of 8.7% (95% CI 6.4, 11.2). We further suggested closing the wound primarily in the low risk group with risk of SSI of 12.3 (95% CI 9.0, 15.5). However, in moderate and high risk groups, primarily closing the wound should be done with caution. Other wound interventions such as wound edge protectors [28], subcutaneous wound drainage [29], or daily wound probing [30] should be considered to apply.

Early detection and treatment of SSI is also important to reduce impact of SSI to a patient, which includes pain, isolation, insecurity, and costs [31]. Knowing risk of SSI can help physician educate, inform, and arrange appropriate infection surveillance protocol. Patients should be advised about symptoms and signs of SSI before discharging home. Patient information sheets or self-assessment questionnaires may be useful [32].

The model was derived with standard methodological analysis including 4 markedly significant variables. Risk stratification was also easy to apply by just counting occurring variables at the end of operation. The internal validation also demonstrated good discrimination and calibration. The final prediction model contained 4 variables that should not result in model optimism with 48 events of SSI as for recommendation by TRIPOD [7]. An internal validation was performed using a bootstrapping technique, which was more appropriated than splitting data when a number of event of interest was small [7]. However, external validation and impact of applying the model to outcomes should be done in the future [33].