Developing a primary tumor and lymph node 18F-FDG PET/CT-clinical (TLPC) model to predict lymph node metastasis of resectable T2-4 NSCLC

This study retrospectively reviewed the charts of 5565 patients examined by ¹⁸F-FDG PET/CT scanning 1–3 weeks before surgery between August 2016 and December 2017 at Shanghai Chest Hospital and identified 192 pulmonary malignancy patients with resectable T2-4 NSCLC. The exclusion criteria were as follows: (1) patients with other than pulmonary malignancy (2616 cases); (2) a history of pulmonary-associated surgical or non-surgical therapy before the ¹⁸F-FDG PET/CT scan (1445 cases); (3) pathologic subtypes other than NSCLC (220 cases); (4) multi-primary tumors (654 cases); (5) an uncertain pathological stage (252 cases); (6) hard to define lymph regions of interest (54 cases); (7) tumor length < 3.0 cm (42 cases); (8) lymph node length < 0.5 cm (2 cases); and (9) lymph nodes without metabolism (88 cases). All patients involved underwent lobectomy combined with systematic hilar (N1) and mediastinal lymph (N2) node dissection within 3 weeks after ¹⁸F-FDG PET/CT examination. The pathological mediastinal lymph node status involved in this study is post-operative pathological N (pN) staging according to the post-operative pathological results. According to the pN staging, the included cases were divided into negative (pN0-1) and positive (pN2) groups. The process of case screening and grouping is shown in Fig. 1. This retrospective study was reviewed and approved by the Institutional Review Board of Shanghai Chest Hospital and the requirement for informed patient consent was waived.

All patients selected in this study underwent the same PET/CT examinations by using the same equipment (Biograph mCT-S PET/CT (64-slice spiral CT), Siemens, Munich, Germany). ¹⁸F-FDG was produced and supplied by Shanghai Atom Kexin Pharmaceutical Co., Ltd. (Shanghai, China), with a pH value of ~ 7.0 and radiochemical purity of > 95%. Before the scan, the patients fasted for at least 6 h and maintained a blood glucose level < 7.8 mmol/L. According to each patient’s weight, the amount of ¹⁸F-FDG injected was based on each patient’s weight according to the standard of 0.10–0.15 mCi/kg. It was injected during a period of calm rest for 45–60 min. The parameters of the CT scan were set with the tube voltage of 120 kV, the tube current adjusted using CARE Dose technology. And the CT images were reconstructed to a 512 × 512 matrix corresponding to a 1 mm pixel size with thickness of 2 mm, 0.98 mm in-plane spatial resolution. The PET scan was performed after the CT scan finished. The collection of PET images was set in 5–6 beds at 2 min per bed position, and then the images were reconstructed to a 200 × 200 matrix corresponding to a 4 mm pixel size with thickness of 3 mm, 4.07 mm in-plane spatial resolution. No low-pass, smoothing filter was applied to the images after reconstruction. The PET images were attenuated by CT data and reconstructed by TrueX + TOF method.

All post-operative pathological sections from patients selected in this study were reviewed by an experienced pathologist (Yichen Han, > 15 years of experience with lung cancer pathologic diagnosis). The pathological mediastinal lymph node status was also recorded.

The target lesions in this study were primary tumors and the lymph nodes with ¹⁸F-FDG uptake. The volume of interest (VOIs) were semi-automatically segmented around the tumor outline to identify the largest cross-sectional area of the target lesions on the fusion PET/CT images. VOIs were further reviewed and corrected by two nuclear radiologists (Gang Huang and Liu Liu. > 15 years of experience) who were blinded to the pathologic or radiologic information. SUV_max > 30% was used as the SUV threshold to determine the final contouring margins of the target. Any different opinions were settled by consensus.

For an accurate diagnosis, more texture features and digital information needed to be extracted from PET/CT images. All data were standardized and normalized to facilitate the statistical analysis of index evaluation values. In total, 2662 features were extracted, including 2436 CT-features of primary tumor and lymph node extracted using the PyRadiomics platform. The features were developed to standardize the calculation of the radiomic feature algorithms and ease the feature extraction process to improve reproducibility of the findings (Griethuysen et al. 2017). Additionally, 216 PET-features of primary tumors and lymph nodes were automatically extracted using the Chang Gung Image Texture Analysis package in MATLAB 2012a (MathWorks Inc., Natick, MA, USA) (Fang et al. 2014). The CT-features were extracted based on the original image and by applying Laplacian of Gaussian and wavelet filters. To extract the PET-features, the SUV values contained within the ROIs were relatively resampled to 64 different values to yield a limited range of values; this was done to reduce the noise and to normalize the images (Yang et al. 2017).

The radiomic features were pre-processed, modeled, evaluated, and validated based on the scikit-learn packages (scikit-learn.org) on the Python platform (Swami and Jain 2013). The purpose of feature selection was to recognize a small set of features that are genuinely associated with response from a big pool with ultra-high dimensions. The 2662 radiomic features were ranked based on their importance by applying the Spearman’s relevance to reduce the dimensions and select the optimized features for radiomic modeling. The top 260 most significant radiomic features were selected for reselection analysis. The Spearman’s relevance of the association among the top 260 radiomic features was established using heat maps (Fig. 2). Least absolute shrinkage and selection operator (LASSO) regression analysis was applied to reselect radiomic features with high levels of multicollinearity, thereby eliminating the possibility of overfitting. LASSO with cross-validation (LassoCV) is usually preferable for high-dimensional datasets. The optimal alpha was selected by LASSO with tenfold cross-validation. As the lambda changed from 10⁻⁵ to 10⁻¹, the number of variables entered into the model was reduced, and the absolute values of the coefficients of the variables declined towards zero (Fig. 3a). The LASSO regression model demonstrated the best predictive performance with maximum AUC, while the alpha was 0.00107 with a lambda of 10^–3 (Fig. 3b). Consequently, 50 highly ranked radiomic features were selected (Fig. 4).

In this study, the features used in the ML algorithms included the 50 radiomic features selected above and seven clinicopathological characteristics. The original datasets were divided into the training and validation set randomly (LN = 283), testing set (LN = 122), and N2-validation set (LN = 57). Based on the three TLPC-, TPC- and LPC-datasets, five independent ML algorithms were trained as classifiers to model the training set, including multi-layer perceptron (MLP), light gradient boosting machine (LightGBM), support vector machine (SVM), gradient boosting decision tree (GBDT), and extreme gradient boosting (XGBoost). To select the best ML algorithms, we typically utilized a six-fold cross-validation on the training set via GridSearchCV method in the hyper-parameter space, with 70% randomly selected to train the models and the remaining 30% used to validate the trained algorithms. XGBoost included 51 estimators with max depth was 6 and the average learning rate was 0.1. Each cross-validation of the ML algorithms might have slightly different optimal parameters. To predict LNM, three prediction models (TLPC, TPC and LPC) were established separately based on the selected clinical factors, tumor PET/CT radiomic features, lymph PET/CT radiomic features, and the combination of the above features using the best ML model implemented in the scikit-learn (version 0.22.1) package. The clinical features selected were age, tumor length, gender, subtype, location, diabetes and smoking. The clinical and radiomics features were input into the prediction models trained to predict LNM in the training cohort. The AUC and the average precision (AP) of the precision-recall (PR) curve were used to identify effective performance quantitatively. The P-value of ROC was assessed using the Delong test.

The clinical characteristics of the 192 patients with 192 primary tumors and 462 lymph nodes enrolled in the training (LN = 283), testing (LN = 122), and N2-validation (LN = 57) sets are summarized in Table 1. There was total 176 lymph nodes histologically positive (LNM+) and 286 lymph nodes histologically negative (LNM-). There’s statistical difference in the training set between LNM+ and LNM− group. There’s no statistical difference in subtype (P = 0.819) and location (p = 0.523) in the testing set between LNM+ and LNM− group. There’s no statistical difference in tumor length (P = 0.668), diabetes (P = 0.862) and smoking (P = 0.544) in the N2-validation set between LNM+ and LNM− group.

The performance of the five ML algorithms for the training and validation set after training is shown in Fig. 5. The mean AUCs of the MLP, LightGBM, SVM, GBDT, and XGBoost algorithms were 0.928, 0.926, 0.960, 0.933, and 0.937, respectively, for the validation set. When we evaluated the ML algorithms’ performance on testing set, the SVM and XGBoost algorithms had the same AUC of 0.93, but SVM had poor sensitivity (0.55), whereas XGBoost had an accuracy of 0.85 (specificity: 0.93, sensitivity: 0.75) and AP reached 0.91 (Fig. 6, Table 2). Therefore, we chose the XGBoost algorithm as the classifier for the prediction models due to its optimal performance.

To compare the effectiveness of TPC, LPC, and TLPC models, we evaluated their results for the testing set (Fig. 7, Table 3). And the performances of the cN staging were also evaluated (Fig. 8). The TLPC model had better values (AUC = 0.93 (95% CI 0.891–0.977; specificity = 0.75; sensitivity = 0.93) than the TPC model (AUC = 0.54; 95% CI 0.378–0.706; specificity = 0.38; sensitivity = 0.59), the LPC model (AUC = 0.82; 95% CI 0.743–0.898; specificity = 0.41; sensitivity = 0.92) and the cN staging (AUC = 0.57; 95% CI 0.574–0.662; specificity = 0.62; sensitivity = 0.52).

To evaluate the predicting performance in N2 status of the three models, we applied them to the N2 validation set. The TLPC model yielded the best results, with AUC of 0.94 (95% CI 0.879–1, sensitivity of 0.97, and specificity of 0.88. Detailed diagnostic performance metrics of the other models are summarized in Fig. 9 and Table 3.