Ultrasound-based radiomics analysis for preoperative prediction of central and lateral cervical lymph node metastasis in papillary thyroid carcinoma: a multi-institutional study

Between January 2019 to June 2019, consecutive patients with thyroid nodules in Fudan University Shanghai Cancer Center (Shanghai, China; training and internal validation cohort), Zhongshan Hospital (Shanghai, China; external validation cohort 1) and Ruijin Hospital (Shanghai, China; external validation cohort 2) were included. The inclusion criteria were as follows: (1) primary surgical resection was performed for the target tumor which had a pathological diagnosis of PTC; (2) preoperative neck US was performed, and the images were recorded and saved as DICOM format; (3) US images meet the requirements for the stored images [15]; (4) patients with complete clinicopathological information. The exclusion criteria included the following: (1) patients with more than one lesion confirmed to be PTC; (2) no history of preoperative therapy (radiofrequency or microwave ablation, neck radiation therapy); (3) patients with only microscopic cervical LNM. After exclusion, a total of 720 patients were enrolled in this study. Four hundred forty-three patients were chronologically divided into two cohorts: the training cohort with 300 patients who were treated between January and April 2019, and the internal validation cohort with 143 patients who were treated between May and June 2019. Two external validation cohorts enrolled 144 and 133 consecutive patients, respectively, using the same criteria. The patients were divided into two groups based on the pathological reports of the lymph node status after CLN dissection.

This retrospective study was approved by the Ethics Committees of all participating hospitals and performed in accordance with the Declaration of Helsinki. The inform consent was waived because of the retrospective nature of the study.

All patients received either total thyroidectomy or lobectomy in accordance with the clinical TNM staging and underwent therapeutic or prophylactic central compartment dissection. Lateral neck LN dissection was performed for patients with FNA proven, or radiologically suspicious LNs that were not suitable for FNA. The resected thyroid and LNs specimens were collected and subjected to pathological examination, including the determination of unifocal or multifocal, and tumor size (maximum diameter of the tumor); the number, size and neck level of the metastatic LNs. Experienced pathologists in each of the three institutions reviewed and validated the pathological results. The pathological results of the LNs were used as the gold standard [15].

The baseline clinical information, including age, gender, thyroglobulin (TG), thyroid stimulating hormone (TSH), thyroglobulin antibody (TGAB), thyroid peroxidase antibody (TPOAB), and cytological findings of FNA were collected from the medical record system and was double checked by the qualified specialists. The results of TG, TSH, TGAB, and TPOAB were detected within one week prior to surgical treatment. According to the previous reports and clinical experience [18, 19], the threshold set for TG, TSH, TGAB, and TPOAB were as follows: TG ≥ 77 ng/ml, TSH ≥ 2.5 ng/ml, TGAB ≥ 100 IU/ml, and TPOAB ≥ 35 IU/ml. Imaging data including US images, US reports and CT reports were obtained from the medical image station.

All patients received neck US examination before surgery. US images were acquired with a Supersonic Aixplorer System using a 15–4 linear-array transducer (SuperSonic Imaging, France) by radiologists with more than six years of experience. The US acquisition parameters were consistent among patients: image depth, 3 cm; focus parallel to the lesion, and gain 53%, and the spatial resolution of axial and lateral was 0.2 mm and 0.4 mm, respectively [20]. The detailed requirements for the US images were described in our previous study [20]. The diagnostic result “central LNM” or “lateral LNM” in the US report was defined as US-reported central/lateral cervical LNM positive. The results of “LNs negative”, “inflammatory LNs” and “small LNs” were categorized as US-reported central/lateral cervical LNM negative [15, 20]. The abnormal US finding suggestive of LNM include the following: (1) round shape; (2) absence of the fatty hilum; (3) microcalcification within the nodes; (4) presence of peripheral flow; (5) heterogeneity with cystic components [21]. LNs that met two or more of the five criteria would be considered positive. All the US images and reports were retrospectively reviewed and validated by independent senior radiologists. The diagnostic sensitivity, specificity and accuracy of neck US examination in the training and validation sets were calculated based upon the US reports [15].

A representative US image was selected, in which the tumor was delineated as the ROI, and was manually segmented by two senior US physicians with more than 6-year-experience in thyroid US imaging who were blinded from the final LNs status. Before feature extraction, the area inside the ROI where the feature extracted was min–max normalized into 0–255 to remove bias, and scaling factors of the effect of different imaging parameters. To avoid the effect of outliers, we calculated the maximum and minimum values after removing outliers for min–max normalization of images. Then, the software “PTC cervical LN metastasis prediction system” developed by the Department of Electronic Engineering, Fudan University [20] was used to input DICOM images after delineation, followed by extraction of image features. (MATLAB R2015b, Mathworks). To evaluate the reproducibility of feature extraction, fifty cases were randomly selected and double-blinded for comparing manual segmentation by two physicians. The interclass correlation coefficient (ICC) was calculated to measure the intra-observer and inter-observer agreement of the radiomics feature extraction. An ICC greater than 0.80 was considered to indicate good agreement.

The detailed procedures of the feature selection and the radiomics signature construction for predicting the central and lateral cervical LNM were described in our previous studies [15, 16, 20], respectively. Briefly, based on the training set, 614 extracted reproducible features were reduced to 19 and 16 features, respectively, using our feature selection model. The selected radiomics features were then combined to build two radiomics signatures (Rad-score). Based on the estimated coefficient, a Rad-score for each case was computed to reflect the risk of cervical central and lateral LNM.

The receiver operating characteristic curve (ROC) of the US-reported LN status, radiomics signature, and radiomics nomogram in each cohort was plotted. The discriminatory performance of the three models was evaluated using the area under curve (AUC). Calibration of the US radiomics nomogram in all four cohorts was assessed by the calibration curve and Hosmer–Lemeshow test. To determine the clinical usefulness of the US radiomics nomogram, decision curve analysis (DCA) was performed by quantifying the net benefits at different threshold probabilities in our entire dataset.

The statistical analyses were performed with R software (version 3.5.3 http://​www.​r-project.​prg), where the package ‘shiny’, ‘foreign’, ‘nomogramFormula’, ‘tmcn’, ‘rms’, ‘set’, ‘glmnet’, ‘rmda’ and ‘DT’ were applied. All other statistical tests were conducted using SPSS software (version 26, Chicago, IL). The categorical and normally distributed continuous variables were presented as frequency (percentage) and mean ± standard deviation (SD), respectively. Categorical variables were compared by χ² test; student’s t-test was used for comparison between normally distributed continuous variables. Rad-scores were presented median (interquartile range), and the potential association of the Rad-scores and LN status in the training and validation cohorts were assessed using a Mann–Whitney U test. The Delong test was employed to compare different AUCs. A two-sided p < 0.05 was considered to indicate statistically significant.

The clinicopathological characteristics of the patients in the entire dataset are presented in Tables 1 and 2. The positive rate in terms of the central LNM in the training and validation cohorts were 34.7%, 32.9%, 48.6%, and 31.6%, respectively; and 9.0%, 10.5%, 10.4%, and 14.3%, respectively, with regard to the lateral LNM. No significant differences in the positive rate were found among the four cohorts (p = 0.058; p = 0.435).

In total, 614 US radiomics features were extracted from the original thyroid US images. Among them, features with nonzero coefficients were selected using our feature selection model [20]. The flowchart of the feature selection is presented in Additional file 1: Fig. S1, and the selected features in the radiomics score calculation formula can be found in Additional file 1: Table S1 and S2. The distribution of the Rad-score for patients with and without central/lateral LNM in the training and validation cohorts were displayed in Tables 1 and 2.

The Rad-score was significantly higher in the central and lateral LNM groups than that in the non-LNM groups in both the training set and the validation sets (p < 0.001, p < 0.001, Tables 1 and 2). The radiomics signature yielded an AUC of 0.839 (cutoff value: − 0.389) for discriminating between central/non-central LNM groups in the training set, and 0.819, 0.799, 0.797, respectively, in three validation sets (Fig. 1). The AUC for differentiating between lateral/non-lateral LNM groups was 0.908 (cutoff value: -0.063) in the training set and were 0.888, 0.796, 0.793, respectively, in three validation cohorts (Fig. 2). The radiomics signatures yielded a better discriminatory performance of cervical LNM than that of the radiologists’ subjective prediction (Tables 3 and 4).

The age (< 45), US-reported central CLN status, and radiomics signature were identified as independent predictive factors for central LNM. The US-reported lateral CLN status and radiomics signature were identified as independent variables for lateral LNM by multivariate logistic regression analysis (Tables 5 and 6). The two radiomics nomograms were constructed by incorporating the independent predictors. In the training sets, the radiomics nomogram showed the highest discrimination between central LNM positive and negative with an AUC of 0.875 (95% CI 0.834–0.915; cutoff value: 0.273; ~ 52 points; Fig. 3a); the nomogram for discriminating between the lateral LNM positive and negative also yielded the greatest AUC (0.938, 95% CI 0.887–0.989; cutoff value: 0.177; ~ 52 points; Fig. 3b), which indicated that nomogram achieved better discriminatory performance than either the US-reported CLN status or the radiomics signature. The favorable discrimination was also observed in the validation cohorts (Tables 3 and 4). The calibration curve of the radiomics nomograms for the probability of central/lateral LNM demonstrated an optimal consistency between the prediction and pathologic observation in the training and validation cohorts (Figs. 4 and 5). The Hosmer–Lemeshow test showed no statistical significance, which suggested no significant deviation from a perfect fit.

The decision curve analyses (DCA) for the US-reported CLN status, the radiomics signature, and the nomograms are illustrated in Fig. 6. The DCA demonstrated that, compared with other models, the nomogram exhibited an optimal net benefit to predict the central LNM for threshold probability within the range of 0–0.98. And the prediction of lateral LNM with radiomics nomogram could benefit more as compared to the radiomics signature and the US-reported lateral CLN status when the threshold probability ranged from 0.04 to 0.98.