Shell feature: a new radiomics descriptor for predicting distant failure after radiotherapy in non-small cell lung cancer and cervix cancer

Our study was conducted at our institution, on two cohorts of patients approved by Institutional Review Board: (1) 48 early stage IA and IB NSCLC patients treated with SBRT from 2006 to 2012 (28 males and 20 females; mean age, 70.58  ±  9.84 years; range, 54–90 years); (2) 52 stage IB-IVA cervix cancer patients without para-aortic node involvement, treated with EBRT and concurrent chemotherapy followed by high-dose-rate ICBT from 2009 to 2012 (mean age, 47.10  ±  11.82 years; range, 26–72 years). The gold standard of this study is the state whether the patient developed distant failure after the treatment, which is represented as a binary outcome: the number 0 denotes not having distant failure while 1 denotes having distant failure. In the NSCLC dataset, the total number of PET slices for each patient varied from 274 to 355, with 2.00 to 5.00 mm slice thickness and 4.0  ×  4.0 mm or 5.0  ×  5.0 mm pixel spatial resolution. Spline-based interpolation has continuous first and second order derivatives and has been shown to generate more accurate and smooth results over several other methods for medical imaging (Meijering et al 2001, Saha et al 2015). Therefore, to achieve a consistent resolution, all slices were interpolated with the smallest slice thickness of 2.0 mm by spline interpolation along the axial dimension for 1D interpolation and spatial resolution of 4.0  ×  4.0 mm by bicubic spline algorithm in the axial plane for 2D interpolation. Detailed information about the interpolation is described in the appendix A. In the CC dataset, all slices were used directly without interpolation since they had the same 5.00 mm slice thickness and 4.0  ×  4.0 mm pixel spatial resolution. Before tumor analysis, the raw PET data were converted to standard uptake values (SUV).

For each patient, slices containing primary tumors were selected for analysis. In the NSCLC cohort, tumors were segmented automatically, with the middle location slice segmented by the object information based interactive segmentation method (OIIS) (Zhou et al 2013) and other slices segmented by the OTSU method. In the CC cohort, the region of interest that incorporated the entire tumor was delineated manually by a radiation oncologist with 4 years' experience and reviewed by another radiation oncologist with 19 years' experience. In the NSCLC cohort, the number of selected slices originally ranged from 5 to 17, and zero padding was used for patients with slice numbers less than 17. Therefore, after interpolation to the smallest slice thickness of 2.0 mm, all patients had 42 slices. Meanwhile, because the greatest in-plane tumor diameter in all the patients' slices was 13 pixels, a patch of 17  ×  17 pixels was cropped around the tumor center in each slice, resulting in a cube size of 17  ×  17  ×  42 for each patient. A volume size of 29  ×  29  ×  40 was used for each patient in the CC cohort. All features were computed on cropped PET cubes.

The shell feature was extracted from the voxels around the tumor boundaries in a series of axial PET slices. The workflow of the shell feature construction is illustrated in figure 1. The top row shows slices of the tumor (outlined in the red windows) in axial sequence (figure 1(a)). First, as displayed in the second row (figure 1(b)), the patches that include the delineated tumor were cropped from the corresponding slices above. The PET intensity values in the delineated tumor were remained while the pixels outside the tumor contour were set to be zero. Furthermore, binary mask images were obtained to represent the specific tumor region by setting inside tumor region as 1 and outside tumor region as 0. A number of grayscale sub-shells $\Psi (t)$ (outlined in yellow, blue and green squares in figure 1(c)) were derived by calculating the difference images between every two adjacent grayscale patches $P(t)$ (outlined in yellow, blue and green rectangles in figure 1(b)) and their corresponding binary masks $M\left(t \right)$. As formulated in equation (1), the sub-shell was then obtained by combining the two difference images together through the Hadamard product. As expected, the sub-shell image was generally the outer region of the tumor. Finally, by adding up the sub-shells $\Psi (t)$ together shown in equation (2), the shell $S\left(k \right)$ (figure 1(d)) was formed and used to represent the holistic heterogeneity of voxels in the boundary of the entire tumor volume.

Zoom In Zoom Out Reset image size

The sub-shell sequence $\Psi (t)$ in figure 1(c) is defined as:

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \Psi \left(t \right)=\left\{\begin{array}{@{}llllllllllllllllllllll@{}} \left(M\left(t \right)-M\left(t-1 \right) \right)\circ \left(P\left(t \right)-P\left(t-1 \right) \right), & t\geqslant 2 \nonumber \\ {\bf 0}, & t=1 \nonumber \end{array} \right.,\nonumber \end{align} \tag{ 1 }$$

where $P(t)$ denotes a grayscale patch sequence shown in figure 1(b) and $M\left(t \right)$ is the matching binary mask image sequence that indicates the region of the tumor, the symbol $\circ$ indicates the Hadamard product and $~t$ is the patch (slice) number. When $t=1$,$\Psi \left(t \right)$ is a zero matrix $0.$ The element in $\Psi \left(t \right)$ is either greater than (when corresponding elements in $M\left(t \right)$ and $M\left(t-1 \right)$ are 0, 1 or 1, 0) or equal to zero (when corresponding elements in $M\left(t \right)$ and $M\left(t-1 \right)$ are 0, 0 or 1, 1). Thus, each sub-shell partially describes the heterogeneous architecture of the tumor edge in an image where the higher SUV value pixels appear brighter. Examples of sub-shells are presented in figure 1.

To represent the heterogeneity of the whole tumor border, for each patient $k$ the shell feature $S(k)$ is constructed by successively accumulating sub-shells together and can be written as:

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle S\left(k \right)=\sum\limits_{t=2}^{n}{\Psi \left(t \right)},\nonumber \end{align} \tag{ 2 }$$

where $n$ is the total slice amount with $n=42$ in the NSCLC cohort and n  =  $40$ in the CC cohort. The strength of the shell feature is the use of a compact, yet comprehensive description that captures a sequence of morphologic patterns across the tumor boundary, such as shape, size, SUV values, and heterogeneities in a simple 2D map (figure 1, bottom row). It is noted that this 2D map as a whole is considered as a shell feature in this work. The vectorized shell (e.g. 1D vector converted from the 2D map) is then used as the input for a support vector machine (SVM) based classifier (section 2.5). This approach is along the same line of voxel/pixel-based methods (Zuluaga et al 2015, Huang et al 2017, Khamis et al 2017) without further explicit calculating handcrafted features as those in a typical radiomics-based method.

Our proposed shell feature was compared with the following five groups of handcrafted features extracted from the whole delineated tumor region: 9 intensity features, 8 geometry features, 12 s order gray level co-occurrence matrix (GLCM) features, 5 high order neighborhood gray tone difference matrix (NGTDM) texture features, and a combination of these four types, for a total of 34. The features are described in table 1 and calculation functions are provided in the appendix B.

To develop our prediction model, a supervised learning method SVM with Gaussian radial basis function (RBF) is employed to classify the patients with and without distant failure into two categories. SVM is a discriminative model that can classify data through a separating hyperplane representing the largest separation margin between two classes and having two parameters penalty factor C and gamma (Suykens and Vandewalle 1999). We used the procedure in the LIBSVM toolbox (Chang and Lin 2011) and conducted the grid search to find the best parameters by minimizing the classification error of SVM as illustrated in literature (Hsu et al 2003). Before being fed to SVM, the vectorized shell was applied by principal component analysis (PCA) to reduce the feature dimension. The reduction process is described in the appendix C. The predictive ability of the shell feature was compared with that of the other five features using ten random trails of 5-fold cross validation on both NSCLC and CC cohorts. Meanwhile, to handle the class imbalance problem, the synthetic minority over-sampling technique (SMOTE) is applied to augment the minority (samples with distant failure) category by creating synthetic examples on the basis of minority class neighborhood distribution (Chawla et al 2002). In this study, 24 (the difference between 12 distant failure positive and 36 distant failure negative) synthetic samples are generated in the NSCLC cohort and 24 (the difference between 14 distant failure positive and 38 distant failure negative) new samples were created in the CC cohort. Accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) were used as evaluation metrics. The code was implemented in Matlab (version R2016a).

The difference in AUC performance between the shell feature and the other features was assessed by the Student's t-test. The difference was considered statistically significant with a P value less than 0.05. The receiver operating characteristic (ROC) curve with 95% confidence interval is presented in figure 2. Statistical analysis was performed with the Matlab statistical toolbox (version R2016a).

Zoom In Zoom Out Reset image size

The demographic and clinical characteristics of patients in the NSCLC and CC cohorts are listed in table 2. No significant difference in distant failure prevalence was observed between the two trials (P  =  0.917). During follow-up time, distant metastases were observed in 25% (12 of 48) of patients in the NSCLC cohort and 26.9% (14 of 52) in the CC cohort after radiotherapy.

The comparison between the shell feature and other features was performed on both NSCLC and CC cohorts through quantitative analysis (table 3) and ROC graphing (figure 2). AUC, sensitivity, specificity, and accuracy were the criteria used in the study. Definitions are given in the appendix D.

The shell feature showed the highest accuracy in predicting distant failure (table 3). In the NSCLC cohort, the shell feature achieved an AUC of 0.82 (95% CI, 0.6632 to 0.9247) with 0.81 sensitivity, 0.80 specificity, and 0.81 accuracy. For the other five features, the best result was observed for the GLCM texture as shown by 0.76 AUC (95% CI, 0.5528 to 0.8905), 0.75 sensitivity, 0.74 specificity, and 0.75 accuracy. Similarly, in the CC cohort the shell feature still achieved the best performance for all metrics, with 0.83 AUC (95% CI, 0.6559 to 0.9212), 0.81 sensitivity, 0.80 specificity, and 0.80 accuracy. These results revealed that the shell feature had more discriminative capacity than the other features. Also, the difference in AUC performance between the shell feature and the other features was found to be significant (P  <  0.005 for both features in both cohorts).

The ROC curves for different feature sets are illustrated in figure 2. Similar results were obtained for NSCLC (figure 2(a)) and CC ((b)). The proposed shell feature, represented by the upper blue curve, is located close to the top left corner of the chart, indicating on average a greater discriminative ability than the other methods.

The discriminative ability is indicated by representative 2D shell maps (figures 3(a) and (b)). The top row shows tumors without distant failure (figure 3(a)) and the bottom row reports those with distant failure (figure 3(b)). Pixels with higher SUV values are indicated in brighter colors, while lower values are shown in darker colors. As evident from the shell maps, distant failure-positive tumors show more heterogeneous boundary expression than the distant failure-negative ones. This finding may be attributed to the more active, varied, and potentially invasive cellular behavior of the tumor in the barrier microenvironment.

Zoom In Zoom Out Reset image size

The overall capability of the shell's classification for the NSCLC and CC cohorts is further illustrated in figures 3(c) and (d), where the rows in the matrices are the vectorized shell's sparse coefficients learned by the dictionary learning method (Gu et al 2014). Given the matrix X (each column is a vectorized shell and the number of the rows is the length of the shell) with class label Y (the binary outcome 0/1 of distant failure state), the task of dictionary learning under the framework of sparse representation is to solve the optimization problem $\left. \{{{D}^{*}},{{Q}^{*}} \right\}=\arg \underset{D,Q}{\mathop{\min }}\vert\vert X-DQX\vert\vert_{F}^{2}+\varphi (D,Q,X,Y)$, where $\varphi (D,Q,X,Y)$ is a discrimination function, D is the synthesis dictionary utilized to reconstruct X, Q is the analysis dictionary applied to encode X. Using the solved D and Q, the sparse coefficient matrix can be obtained by W  =  QX with each column being a sparse coefficient corresponding to its vectored shell of a patient, which is a projection of the original training data (the input vectorized shell) to the learned dictionary that includes the discriminative information. Clustering characteristics can be seen on both cohorts, with features of the same class showing similar representation and features of different classes displaying distinct representations. In the horizon row of figures 3(c) and (d), the sparse coefficients in DF-negative has higher values in the first half and lower values in the second half, while the sparse coefficients in DF-positive has lower values in the first half and higher values in the second half.

To further evaluate the shell's predictive ability, another cohort of 23 early stage IA and IB NSCLC inoperable patients underwent SBRT from 2012 to 2016 (12 males and 11 females; mean age, 73.30  ±  9.58 years; range, 55 to 91 years) in our institution was used for an independent validation. The distant metastasis was observed in 30.4% (7 of 23) of patients. The demographic and clinical characteristics of patients are listed in table 4. There are no significant differences between the validation and primary cohorts (the cohort of 48 patients) in the characteristics (P  =  0.320), either within the distant failure positive cohort (P  =  0.336) or in the distant failure negative cohort (P  =  0.792), which justified its use as a validation cohort. The primary cohort of 48 NSCLC patients with 5-fold cross validation were used for training and the trained model was then applied on the validation cohort.

The performance of shell feature in the validation set was also compared with other features. The experiment results are shown in table 5. The shell feature still outperformed the other features for distant failure prediction in the validation cohort. The AUC achieved by the shell feature was 0.79 (95% CI, 0.6559 to 0.9212) with 0.71 sensitivity, 0.87 specificity, and 0.83 accuracy, while the highest AUC achieved by the other features was 0.70 (95% CI, 0.4079 to 0.8442) with 0.71 sensitivity, 0.62 specificity, and 0.65 accuracy, respectively.

Intensity features were calculated on the intensity histogram of SUV value and include 9 features:

• (1)  
  Minimum;
• (2)  
  Maximum;
• (3)  
  Mean;
• (4)  
  Stand deviation;
• (5)  
  Sum;
• (6)  
  Median;
• (7)  
  Skewness describes the degree of distribution asymmetry around its mean:

  $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm Skewness}=E\left[ {{\left(\frac{X-\mu }{\sigma } \right)}^{3}} \right],\nonumber \end{align} \tag{ B.1 }$$

  where X denotes SUV value, $\mu$ is the mean and $\sigma$ is the stand deviation.
• (8)  
  Kurtosis describes the flatness or the spikiness of the signal:

  $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm Kurtosis}=E\left[ {{\left(\frac{X-\mu }{\sigma } \right)}^{4}} \right].\nonumber \end{align} \tag{ B.2 }$$

  Where X denotes SUV value, $\mu$ is the mean and $\sigma$ is the stand deviation.
• (9)  
  Variance;

Geometry features describe the shape, size, or relative position of the tumor. The major diameter is defined as the major axis length or longest diameter, while the minor diameter is the shortest diameter. Eccentricity is the aspect ratio, defined as the ratio of the length between the major and the minor axis. These features are calculated by the regionprops function in Matlab.

• (1)  
  Volume;
• (2)  
  Major diameter is the major axis length or longest diameter;
• (3)  
  Minor diameter the shortest diameter;
• (4)  
  Eccentricity is the ratio of the length between the major and the minor axis;
• (5)  
  Elongation;
• (6)  
  Orientation the angle between the x-axis and the major axis of the ellipse that has the same second moments as the region;
• (7)  
  Bounding Box Volume the smallest rectangle containing the region;
• (8)  
  Perimeter the distance around the boundary of the region;

The gray level co-occurrence matrix (GLCM) is a square matrix with the number of rows and columns equaling the quantized gray level denoted by ${{N}_{g}}$. Each element $p(i,j)$ in GLCM represents the number of times a pixel of gray level $i$ occurs with a neighbor pixel of gray level $~j$ in the image at a particular displacement distance and angle. We used histograms with 64 bins and constructed GLCM using 3D analysis of the tumor region with 26 neighboring voxels and 13 directions of the 3D space.

• (1)  
  Energy describes image homogeneity and a higher energy value indicates a more homogeneous image,

  $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm Energy}=\sum\limits_{i=0}^{{{N}_{g}}-1}{\sum\limits_{j=0}^{{{N}_{g}}-1}{p{{(i,j)}^{2}}}},\nonumber \end{align} \tag{ B.3 }$$
• (2)  
  Entropy describes the randomness of the image intensity distribution,

  $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm Entropy}=-\sum\limits_{i=0}^{{{N}_{g}}-1}{\sum\limits_{j=0}^{{{N}_{g}}-1}{p\left(i,j \right)\log \left(\,p\left(i,j \right) \right)}},\nonumber \end{align} \tag{ B.4 }$$
• (3)  
  Correlation measures the linear dependency of gray levels on those of either neighboring voxels or specified points,

  $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm Correlation}=\frac{\sum\nolimits_{i=0}^{{{N}_{g}}-1}{\sum\nolimits_{j=0}^{{{N}_{g}}-1}{\left(ij \right)\cdot p\left(i,j \right)-{{\mu }_{x}}{{\mu }_{y}}}}}{{{\sigma }_{x}}{{\sigma }_{y}}},\nonumber \end{align} \tag{ B.5 }$$
• (4)  
  Contrast is the local variations within an image,

  $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm Contrast}=\sum\limits_{k=0}^{{{N}_{g}}-1}{{{k}^{2}}}\left\{\sum\limits_{i=0}^{v}{\sum\limits_{j=0}^{{{N}_{g}}-1}{p\left(i,j \right)~\left| i-j \right|=k}} \right\},\nonumber \end{align} \tag{ B.6 }$$
• (5)  
  Texture Variance the variation around the mean value,

  $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm Variance}=\sum\limits_{i=0}^{{{N}_{g}}-1}{\sum\limits_{j=0}^{{{N}_{g}}-1}{{{(i-{{u}_{x}})}^{2}}\cdot p(i,j)}+\sum\limits_{i=0}^{{{N}_{g}}-1}{\sum\limits_{j=0}^{{{N}_{g}}-1}{{{(i-{{u}_{y}})}^{2}}\cdot p\left(i,j \right)}}},\nonumber \end{align} \tag{ B.7 }$$
• (6)  
  Sum-Mean is defined as,

  $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm Sum{\text {--}}Mean}=\sum\limits_{i=0}^{{{N}_{g}}-1}{\sum\limits_{j=0}^{{{N}_{g}}-1}{\left(i+j \right)\cdot p\left(i,j \right)}},\nonumber \end{align} \tag{ B.8 }$$
• (7)  
  Inertia measures the local variation between a voxel and its neighbors,

  $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm Inertia}=\sum\limits_{i=0}^{{{N}_{g}}-1}{\sum\limits_{j=0}^{{{N}_{g}}-1}{{{(i-j)}^{2}}\cdot p(i,j)}},\nonumber \end{align} \tag{ B.9 }$$
• (8)  
  Cluster Shade measures matrix skewness,

  $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm Cluster}\,{\rm Shade}=\sum\limits_{i=0}^{{{N}_{g}}-1}{\sum\limits_{j=0}^{{{N}_{g}}-1}{{{\left(i+j-{{u}_{x}}-{{u}_{y}} \right)}^{3}}p\left(i,j \right)}},\nonumber \end{align} \tag{ B.10 }$$
• (9)  
  Cluster tendency describes asymmetry,

  $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm Cluster}\,{\rm Tendency}=\sum\limits_{i=0}^{{{N}_{g}}-1}{\sum\limits_{j=0}^{{{N}_{g}}-1}{{{\left(i+j-{{u}_{x}}-{{u}_{y}} \right)}^{4}}p\left(i,j \right)}},\nonumber \end{align} \tag{ B.11 }$$
• (10)  
  Homogeneity describes closeness of the elements in $p(i,j)$ to the diagonal,

  $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm Homogeneity}=\sum\limits_{i=0}^{{{N}_{g}}-1}{\sum\limits_{j=0}^{{{N}_{g}}-1}{\frac{p(i,j)}{1+|i-j|}}},\nonumber \end{align} \tag{ B.12 }$$
• (11)  
  Max-Probability is defined as,

  $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm Max{\text {--}}Probability}=\underset{i,j}{\mathop{\max }}\,p\left(i,j \right),\nonumber \end{align} \tag{ B.13 }$$
• (12)  
  Inverse Variance is defined as,

  $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm Inverse}\,{\rm Variance}=\sum\limits_{i=0}^{{{N}_{g}}-1}{\sum\limits_{j=0}^{{{N}_{g}}-1}{\frac{p(i,j)}{1+{{\left(i-j \right)}^{2}}}}}.\nonumber \end{align} \tag{ B.14 }$$

The neighborhood gray tone difference matrix (NGTDM) is a high order texture and which related to human perception. It represents a difference in grayscale between pixels with a certain gray scale and the neighboring pixels. Let $P(i)$ represents the summation of the gray-level differences between all voxels with gray level i and the average gray level of their 26 neighbors in 3D space. ${{N}_{g}}$ represents the quantized gray level in V, and ${{({{N}_{g}})}_{ef\,\!f}}$ is the effective number of gray-level in V.

• (1)  
  Coarseness is defined as,

  $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm coarseness}=\sum\limits_{i=0}^{{{N}_{g}}-1}{{{n}_{i}}P\left(i \right)},\nonumber \end{align} \tag{ B.15 }$$

  where ${{n}_{i}}=\frac{{{N}_{i}}}{N}$, N_i is the number of voxels with gray level i in V and N is the total number of voxels in V.
• (2)  
  Contrast is defined as,

  $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm contrast}=\left[ \frac{1}{{{N}_{g}}({{N}_{g-1}})}\sum\limits_{i=0}^{{{N}_{g}}-1}{\sum\limits_{0}^{{{N}_{g}}-1}{\left. {{n}_{i}}{{n}_{j}}{{\left(i-j \right)}^{2}} \right]}}\left[ \frac{1}{N}\sum\limits_{i=0}^{{{N}_{g}}-1}{P\left(i \right)} \right. \right],\nonumber \end{align} \tag{ B.16 }$$

  where ${{n}_{i}}=\frac{{{N}_{i}}}{N}$, ${{n}_{j}}=\frac{{{N}_{j}}}{N}$, N_i, N_j are the number of voxels with gray level i and j in V and N is the total number of voxels in V.
• (3)  
  Busyness is defined as,

  $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm busyness}=\frac{\sum\nolimits_{i=0}^{{{N}_{g}}-1}{{{n}_{i}}P(i)}}{\sum\nolimits_{i=0}^{{{N}_{g}}-1}{\sum\nolimits_{0}^{{{N}_{g}}-1}{\left. \left(i{{n}_{i}}-j{{n}_{j}} \right) \right]}}},\nonumber \end{align} \tag{ B.17 }$$
• (4)  
  Complexity is defined as,

  $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm complexity}=\sum\limits_{i=0}^{{{N}_{g}}-1}{\sum\limits_{i=0}^{{{N}_{g}}-1}{\frac{\left| i-j \right|\left[ {{n}_{i}}P(i)+{{n}_{j}}P(\,j) \right]}{N({{n}_{i}}+{{n}_{j}})}}},\nonumber \end{align} \tag{ B.18 }$$
• (5)  
  Texture Strength is defined as,

  $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm strength}=\frac{\sum\nolimits_{i=0}^{{{N}_{g}}-1}{\sum\nolimits_{0}^{{{N}_{g}}-1}{\left. ({{n}_{i}}+{{n}_{j}}){{\left(i-j \right)}^{2}} \right]}}}{\sum\nolimits_{i=0}^{{{N}_{g}}-1}{P\left(i \right)}}.\nonumber \end{align} \tag{ B.19 }$$