Venous thromboembolism in COVID-19 patients and prediction model: a multicenter cohort study

We compiled data for adult patients (age 18 years or older) that included baseline demographics, laboratory results, and in-hospital events, including all-cause mortality of COVID-19 patients from March 1, 2020, to the end of December 2020. All patients (with and without VTE events) were included (Fig. 1). For each patient, a total of 85 variables (Additional file 1: Table S1) from six categories were extracted, including baseline demographics, presenting vital signs, past medical history (abstracted using standard-text variables, International Classification of Diseases–Tenth Revision (ICD-10) and Current Procedural Terminology codes), social history, admission reasons, pre-admission and in-hospital medications, hospital course, laboratory values, electrocardiogram, and imaging studies (magnetic resonance imaging (MRI), computerized tomography scan, ultrasounds). Variables in our study included: personal information (age, sex, race/ethnicity, body mass index (BMI), social history), hospital summary (hospital length of stay (LOS), intensive care unit (ICU) admission and LOS, use of oxygen devices, intubation status), laboratory values (white blood cell (WBC) counts, D-dimer, ferritin, LDH, lactate, C-reactive protein (CRP), and so on), past medical history, vital signs, and in-hospital prophylactic and therapeutic anticoagulation therapy. Since COVID-19 can cause VTE in patients following discharge, we followed patients after their initial hospital discharge for readmission and development of VTE. Accordingly, patients with one-time admission and readmissions, with or without thromboembolism events, were considered when building prediction models.

The primary outcome was in-hospital VTE events, including acute deep vein thrombosis (DVT) and pulmonary embolism (PE) identified by ICD-10 codes (Additional file 1: Table S2), venous Doppler ultrasounds, ventilation-perfusion scan, and computed tomography angiography (CTA) of the chest. In-hospital outcomes (Table 1) included mortality, and hospital and ICU LOS.

As part of exploratory data analysis, the distribution of all the variables was plotted. Most laboratory values were either left or right-skewed. Multiple variables could be highly correlated with each other and potentially result in interactions in the process of model building. For example, both neutrophil and lymphocyte counts comprise the neutrophil–lymphocyte ratio. Likewise, BUN and creatinine comprise the BUN-creatinine ratio, which is a parameter that could indicate different types of acute kidney injury; for example, the BUN-creatinine ratio > 20 suggests pre-renal acute kidney injury. Therefore, Spearman’s rho was performed. Twenty-three groups of variables that were highly positively or negatively correlated based on Spearman’s coefficient of more than ± 0.7 (Additional file 1: Table S5A) were aspartate aminotransferase (AST) and alanine transaminase (ALT), creatinine and BUN, maximum (max) B-type natriuretic peptide (BNP) and initial BNP, max CRP and initial CRP, max ferritin and initial ferritin, max D-dimer and initial D-dimer, neutrophil–lymphocyte ratio and neutrophils, max neutrophils and minimum lymphocyte, history of VTE, DVT and PE, systolic blood pressure and diastolic blood pressure, inpatient therapeutic anticoagulation and inpatient prophylactic anticoagulation and so on. Therefore, we downsized the variables; for example, neutrophil and lymphocyte alone were analyzed in the model building rather than the neutrophil–lymphocyte ratio. Likewise, BUN and creatinine alone were included rather than the BUN-creatinine ratio; the history of VTE was used rather than its components (DVT and PE) (Additional file 1: Table S5B). When building models, we used lab values on admission rather than the peak or lowest values as we aimed to build a prediction model which can assist physicians in predicting VTE in COVID-19 patients on admission based on the available data. The PCA was performed to reduce the dimensions used to predict VTE events. Patients without missing data (N = 1443) from the cohort were included in the PCA. A total of 32 continuous variables were included in the PCA. In the scree plot, the 1st component explained only about 16% of variations of the data, and only 24.6% of the variations were explained by the first two components (Additional file 1: Table S3 and Fig. S1). Therefore, the PCA was deemed not helpful in reducing the dimensions in our analysis. For both continuous and categorical variables, we further performed univariate analysis using the R packages (Additional file 1: Table S4).

The cohort was randomly split into the training set and test set (70:30) multiple times. We compared four models in their predictive accuracy for detecting VTE events and mortality:

The most significant variables of each model are shown in Table 2. For MLR and LR, the significant variables were selected based on the p-value of < 0.05; for decision tree and random forest, they were based on the Gini index. MLR was eliminated as it is not ideal for categorical variables. The decision tree has worse accuracy than a random forest but provides interpretability. Our decision tree was firstly split by the root node as therapeutic anticoagulation as inpatient, followed by leaf nodes of BUN (< 20, 20), hospital LOS (< 20, 20), Age (< 91, 91), race (White, non-White), D-dimer (4740 ng/mL, < 4740 ng/mL), history of VTE, and D-dimer (2170 ng/mL, < 2170 ng/mL) (Additional file 1: Fig. S2). Whereas random forests are an ensemble of decision trees that solve the overfitting of the decision tree as the predictions are based on an average of all trees. On the other hand, loss of interpretability is one of the limitations of the random forests. Both decision trees and random forests handle continuous and categorical variables that best analyze our cohort. Across all models, D-dimer was the most significant variable for MLR, LR, and decision tree models. Other common variables across the models include VTE history, inpatient therapeutic anticoagulation, requirement for oxygen devices such as high-flow nasal cannula, non-rebreather mask, and mechanical ventilation, heart rate, BUN, and so on. The four models were compared, as shown in Table 3, to analyze predictive ability in diagnosing COVID-19 associated VTE. Random forest performed the best among all in terms of R-square (R²), misclassification rate, and receiver operating characteristic (ROC) curve.

Random forest model consisted of 22 variables (significance in order): D-dimer, inpatient therapeutic anticoagulation therapy, platelet count, BUN, age, WBC, systolic blood pressure, lymphocytes, ALT, potassium, BNP, CRP, creatinine, LDH, neutrophils, heart rate, total bilirubin, AST, diastolic blood pressure, prior history of VTE, ferritin, and oxygen saturation on admission. Electrolytes, renal function, blood pressures, hepatic enzymes, and inflammatory markers were indicators of VTE risks. The evaluation of the performance and confusion matrix of the four models in training and the validation process is shown in Table 3. The R² of the random forest model for the training and validation set was 58.87% and 18.76% (p < 0.0001); the area under the ROC curve was 0.83 (Fig. 2). We set a cutoff of 0.1 for the generation of sensitivity and specificity. The random forest model had a sensitivity of 0.68 and a specificity of 0.82. In our cohort, the classification was skewed; therefore, the default threshold (0.5) cannot represent an optimal interpretation of the predicted probabilities. Effectively, our goal was to provide a robust model for clinicians to identify COVID-19 patients at risk for VTE early in the hospital course and assist in deciding between therapeutic versus prophylactic anticoagulation management. In the validation set, the model showed that it was good at predicting the absence of a venous event more than the presence of a venous event. The negative predictive value (NPV) and positive predictive value (PPV) of the model for the validation set were 0.97 and 0.26. Due to the low prevalence of VTE in the population, the F1 score of the model was calculated as 0.35.