Clinical applications of machine learning in the survival prediction and classification of sepsis: coagulation and heparin usage matter

Nonparametric methods were utilized to test the differences in features among subgroups when the data violated the assumptions of normal distribution and homoscedasticity. Two nonparametric tests, Kruskal–Wallis and Jonckheere-Terpstra were both utilized, and the higher p value was selected with respect to each comparison. Otherwise, T-test, F-test, and one-way analysis of variance (ANOVA) were conducted accordingly. Additional clinical and laboratory test results are shown in Additional file 1: Table S1 in the Supplemental Material. The p values for the association between features and survival were calculated using nonparametric tests on k-independent samples.

As an unsupervised ML technique, the K-means clustering method was applied to identify the sepsis clusters in MIMIC datasets [57]. The advantages of this clustering method are its fast speed and less parameters needed. To realize the calculation, the acquired features were taken as direct input and the data were automatically aggregated by the distance calculation. An optimal number of groups (k) was determined by compromising between the methods of elbow [58] and the silhouette score [52, 59]. Upon data clustering, PCA was utilized to reduce the data dimensionality to three dimensions to facilitate visualization. Nonparametric tests were used to test the differences among the detected groups.

Since Hinton and Salakhutdinov [60] proposed a multi-level Boltzmann machine based on a probability graph model in 2006, deep learning has gradually become the leading tool in the field of image processing and computer vision. CNN [54, 61] is one of the prominent deep learning algorithms, with a wide range of applications in various fields and an excellent performance in classification tasks [62]. In addition, advancements in numerical computing equipment further promoted CNN’s representational learning ability.

This work proposes a CNN-based survival rate prediction model to predict sepsis patients’ survival rate. The CNN model contains seven convolutional layers, of which the first six layers use the rectified linear unit (ReLU) as the activation function, and the last one utilizes Sigmoid. Convolution layers extract features extraction, and the activation function adds nonlinear factors. ReLU largely solves the gradient vanishing problem when the model optimizes the deep neural network [63]. The Sigmoid activation function serves to transform the probabilities into the output suitable for binary classification problems. The feature map size for each layer is shown in Fig. 1B.

Quadratic fitting function method [64, 65], also known as function simulation or interpolation function method, is recognized as a classical and effective optimization method. To adapt to complex environments, a multidimensional quadratic fitting function [66, 67] was proposed. However, the fitting effect is poor for nonlinear data. To solve this problem, a multivariate quadratic fitting function with double coefficients was proposed to adapt to multi-dimensional nonlinear data for prediction of the survival probability in the current work.

This model considers eleven features, including the most valuable parameters in the SOFA score system that indicate the organ function and two features acquired in Blood Gas Analysis (pH and lactate) critical to estimating the septic shock. More precisely, the considered futures are Creatinine, Hemoglobin, the International standardized ratio of prothrombin time (INR-PT), Lymphocytes, Neutrophils, Platelet Count, Partial Thromboplastin Time (PTT), White Blood Cells, Lactate, Bilirubin, and pH. These features indicate the sepsis severity, with several of them correlating.

First, the data were normalized. To avoid zero minimum values in the normalization process, the formula \({x}^{*}=\frac{x-\mathrm{min}*0.99}{\mathrm{max}-\mathrm{min}}\) is selected. In this formula, x is the element before normalization, x^* is the normalized element, and max and min are the maximum and minimum values of a feature, respectively. The values of the 11 features are regarded as independent variables \({x}_{i},i\in \{\mathrm{1,2},\dots ,11\}\). Survival and death probabilities are considered as a two-dimensional dependent variable, i.e., y = (0,1) or (1,0). The DCQMFF model is defined as:where \(\frac{{e}^{{y}_{1}}}{{e}^{{y}_{1}}+{e}^{{y}_{2}}},\frac{{e}^{{y}_{2}}}{{e}^{{y}_{1}}+{e}^{{y}_{2}}}\) represents the probability of survival and death, respectively (note, \(\frac{{e}^{{y}_{1}}}{{e}^{{y}_{1}}+{e}^{{y}_{2}}}+\frac{{e}^{{y}_{2}}}{{e}^{{y}_{1}}+{e}^{{y}_{2}}}=1\)). Double the coefficients can help avoid over fitting caused by a fast dimensionality reduction, thus improving the model’s generalization ability.

The processed data were divided into training data and test data according to a 7:3 ratio. To prevent class imbalance, negative cases were up-sampled by means of replication, random generation according to the median of negative cases features, and adding random noise to the cases to keep the proportion of positive and negative cases nearly equal [68]. Then, the model was trained on the training data and verified using the test data. This procedure processes only the 11 aforementioned features to predict the survival probability. Receiver operating characteristic curve (ROC) [69] was used to evaluate the effectiveness of the CNN and DCQMFF prediction models. The ROC curve is created by plotting the true positive rate (TPR) with respect to the false positive rate (FPR) at various threshold settings and depicts a trade-off between sensitivity and specificity. Thus, the curve summarizes the binary classifier’s performance by combining the confusion matrices at all threshold values. The area under the ROC curve (AUC) measures the classifier’s ability to distinguish between positive and negative classes. The closer the AUC to 1, the better the model at distinguishing the two classes. Finally, accuracy, precision, recall, and F₁-score are four popular metrics for evaluating the performance of classification methods. The CNN and DCQMFF models’ prediction results are compared to those of the random forest, logistic regression, lasso regression, and other methods considered by Chicco et al. [41] (Table 1).

The survival rate curves were calculated, and the statistical analysis of different phenotypes was performed for dataset. Nomograms are prediction models that estimate an individual’s survival by considering multiple clinical variables and their interdependence. Thus, nomograms can provide an overall probability of a specific outcome for an individual patient, offer a more accurate prediction than conventional staging or scoring systems, and, accordingly, improve personalized decision-making in sepsis therapy. The survival nomogram was established using R software by integrating age and other features.

The proposed models were run in a hardware environment comprising Intel(R) Core (TM) i5-6200U CPU @2.40 GHz, with 8 GB memory. The workstation’s operating system was Windows 10. The integrated development environment and the deep learning symbolic library were PyCharm-Python 3.8 and PyTorch 1.6.0, respectively.

In MIMIC-III, there were totally 2902 cases of sepsis, 531 sepsis patients were excluded due to the lack of some blood test results. The rest were divided randomly into a training set and a test set at a ratio of 7:3. One thousand six hundred and sixty-one cases were assigned into the training set and 710 into the test set. The 35 blood tests were used in K-means and CNN methods. Length of hospital stay was omitted. In MIMIC-IV, there were 12657 cases of sepsis. To balance the number of dead and surviving cases, more than 3000 cases with the 35 blood test results were extracted and served as validation data set for the two proposed prediction models.

Survival prediction enable detect cases of high mortality probability. However, it is difficult to determine the features which lead to death with the survival prediction tools. Thus, the sepsis population was clustered using K-means clustering, and the obtained groups’ phenotype features were analyzed. The elbow and silhouette score methods (Fig. 2A, B) indicated the presence of four clusters, i.e., K = 4 was selected. Upon the K-means classification, a 3D PCA plot was generated (Fig. 2C). The data set was phenotyped by the K-means method into four clusters including C_1 with 211 cases in the training set and 90 in the test set, C_2 with 1215 and 520, C_3 with 46 and 19, and C_ 4 with 189 and 81.

Among the four clusters, Cluster C_2 has the highest survival rate (Fig. 2D). In accordance, this cluster exhibited the lowest SIC and SOFA score (Fig. 2E ,F), further validating the prediction method. C_1 also has a high survival rate. Patients in this cluster are characterized by a low white blood cell count (Fig. 3A) and neutrophil proportion (Fig. 3B) but the highest lymphocyte proportion (Fig. 3C). C_4 is identified as septic patients with abnormal coagulation and had the worse prognosis, characterized by slightly prolonged PTT. C_3 is identified by significantly prolonged PTT (Fig. 3D), high SIC, and higher heparin-using proportion (Fig. 3E) among its patients than those from other clusters. The early mortality rate of patients in C_3 is high but with a better long-term survival rate than those in C_4. Other features of the 4 clusters were plotted in Additional file 1: Figure S1 and S2. The nomograms of each cluster are also established for reference (see Additional file 1: Figure S3).

Table 2, 3 list the features for which the nonparametric tests found the most significant differences in training and test sets respectively. The top ten heterogeneous features shared in the training and test datasets are PTT, neutrophil percentage, PT, INR-PT, lymphocyte percentage, white blood cell count, platelet count, mean corpuscular hemoglobin concentration (MCHC), albumin, and red blood cell count. The survival nomograms were generated for all cases to clarify the relationship between features and death risk using the 35 features and survival information of all 2371 patients in MIMIC-III (Fig. 4).

A multivariate approach to predict mortality outcomes of sepsis patients was utilized. The CNN based model was tested using all 35 blood tests, whereas the DCQMFF prediction model used only 11 blood tests. The obtained ROC curves for CNN (Fig. 5A) and DCQMFF (Fig. 5B) models on training, test, and validation sets are shown. The ROC curves for training, test, and validation sets are virtually smooth, suggesting that an overfit is unlikely, the predictive partition analysis verified that the blood tests are strong predictors of sepsis patients’ status. For the CNN model, the AUC scores for the training, test, and validation sets are 0.953, 0.947, and 0.909, respectively. All of the AUCs are close to 1, indicating that the proposed survival prediction model has a good performance in distinguishing the 28-day survivals of sepsis patients. Figure 5B shows the results for the DCQMFF prediction model. While not as good as CNN, DCQMFF performs well using 11 features, with the AUC values for the training, test, and validation sets equal to 0.896, 0.885, and 0.849, respectively. A demo analysis with DCQMFF-based application platform is shown in Fig. 5C.

The traditional survival prediction methods utilized the same 35 features, and the obtained AUC values for the test data are 0.533, 0.604, and 0.567 for Random Forest, Logistic Regression, and Lasso, respectively. The SOFA score is generally applied in the ICU to access multi-organ dysfunction or failure, which is calculated based on PaO2/FiO2, platelets count, bilirubin level, cardiovascular hypotension, Glasgow Coma Scale (GCS), and creatinine level. When the SOFA score exceeds 12, the mortality surpasses 50% [13]. As shown in Fig. 5D, the prediction SOFA score reached an AUC of 0.807, with the lower and upper bounds equal to 0.783 and 0.823, respectively.

Next, the survival prediction performance of the methods proposed by Chicco et al. [41] with age, sex, and septic episode number alone was compared using MIMIC-III as training and test data and MIMIC-IV as validation data. The AUC values for the test data are 0.586, 0.574, 0.562, 0.541, 0.586 and for the validation data 0.642, 0.574, 0.713, 0.689, 0.676 for radial SVM, Gradient boosting, Bayes, Linear regression, and Linear SVM methods, respectively. Although the proposed methods achieved good results regarding major indicators, the true negative rate (TNR) and AUC were low. This issue was likely caused by the class imbalance. Both the CNN and DCQMFF models exhibited outperformed these methods, with an accuracy of 83.37% and AUC of 0.908 for the CNN model and an accuracy of 77.5% and AUC of 0.849 for the DCQMFF on the validation data (Table 1).