CNN-based survival model for pancreatic ductal adenocarcinoma in medical imaging

Three independent cohorts were used in this study. Cohort 1 consists of publicly available pretreatment CT scans of 422 Non-small cell lung cancer (NSCLC) patients [7]. Cohort 2 has 68 resectable pancreatic adenocarcinoma (PDAC) patients collected from a local hospital from 2008 to 2013. Cohort 3, which is the test data, consists of 30 resectable PDAC patients enrolled in another independent hospital site from 2007 to 2012 [3]. For all the patients in these three independent cohorts, CT scans, annotations (contours) of tumor performed by radiologists, and survival data were available. For PDAC patients, the CT scans were preoperative contrast-enhanced images of resectable patients, and the survival data was collected from the date of surgery until death. CT images from all three cohorts were read from DICOM file without further processing. As CT scans were from different institutions, the image acquisition protocol information (e.g., exact contrast bolus volume, timing, and injection rate) was not consistent over the time period. The institutions’ Research Ethics Boards approved these retrospective studies and all methods were carried out in accordance with relevant guidelines and regulations.

A CNN architecture with six-layered convolutions (CNN-Survival) was trained as shown in Fig. 1. Input images have dimensions of 140 × 140 × 1 (grayscale), which contain the CT images within the manual contours of the tumors (example shown in Fig. 2). All pixels outside of the contoured region were 0 in the 0 to 255 grayscales. All convolutional layers have kernel size of 3 × 3 with 32 filters following by Batch Normalization layers (BN). The first Max Pool layer has pool size of 2 × 2, and the latter two Max Pool layers have pool size of 3 × 3. Through the Max Pool layers, number of trainable parameters was significantly reduced. To avoid overfitting with this small sample size, dropout layers were added after every two convolutional layers with dropout rate at 0.5. Finally, passing through the flatten and dense layer, images were converted into 19 features and finally, survival probabilities for a given time t were calculated.

To better fit the distribution of survival data, a modified loss function, proposed by Gensheimer et al. [11], was applied to the CNN architecture (Eq. 1).

In Equation 1, \( {h}_j^i \) is the hazard probability for individual i during time interval j. r stands for individuals “in view” during the interval j (i.e., survived in this period) and d means a patient suffered a failure (e.g., death) during this interval [11]. As it can be seen from Equation 1, the left part penalizes if the model gave low hazard for failure (e.g., death), while the right part penalizes if the model gave high hazard for a survived case. The overall loss function is the sum of the losses for each time interval [11].

Training a CNN-based survival model needs to finetune a large number of parameters. Given this CNN architecture, there were 73,587 trainable parameters. As such, the larger dataset, cohort 1, was used to pretrain the network. In Cohort 1, 422 patients had 5479 slices containing manually contoured tumor regions. However, the region of interest (ROI) on some of the slices were too small (e.g., less than 250 pixels) to be fed as input to the CNN (shown in Fig. 3). To mitigate this, we ranked slices using their ROI size and pixel intensity and picked the top 2500 slices. This ensured the minimum ROI size of 250 pixels.

These 2500 slices were fed into the proposed CNN model without augmentation. After training the initial model at learning rate 0.0001 for 50 epochs, all the weights in the pretrained model were frozen except for the final dense layer. Next, 68 patients of Cohort 2 were used to finetune the dense layer (containing 627 parameters) for 20 epochs with learning rate of 0.0001 without augmentation. The finetuning was necessary since Cohort 1 and Cohort 2 have CT images from two different types of cancers (lung and pancreas cancer, respectively) with different survival patterns. After 20 epochs of finetuning in Cohort 2, the final model was tested in Cohort 3. The prognosis performance was measured by two metrics: concordance index (CI) [36] and index of prediction accuracy (IPA) [37]. CI is calculated using Equation 2. where the indicator function 1_a < b = 1 if a < b, and 0 otherwise. T_i is the survival time for subject i. | \( \mathcal{E} \) | is the number of edges in the order graph. f(x_i) is the predicted survival time for subject i by model f. Under this formula, concordance index (CI) is the probability of concordance between the predicted and the observed survival [36]. IPA is a recently proposed performance measure for binary and time to event outcomes accounting for both discrimination and calibration, and can identify harmful models as well [37]. IPA of 100% indicates a perfect model, and harmful models will have IPA < 0 [37].

In order to systematically compare the performance of CNN-Survival with CPH models and rule out the confounding variable, we built two additional models using CPH. The first model (Model 1: Radiomics features + LASSO-CPH) is a traditional radiomics-based CPH model, which used 1428 2D radiomics features extracted from the manually contoured regions using PyRadiomics library [2] (version 2.0). A LASSO-CPH [38] feature reduction method was used to find prognostic radiomic features in the training cohort (Cohort 2), which were then tested in the test cohort (Cohort 3). The second model (Model 2: Transfer learning features + LASSO-CPH) was trained using the 19 transfer learning features extracted from the last dense layer of the CNN-Survival model. Similar to Model 1, a LASSO-CPH method was used to select prognostic features in Cohort 2 and test them in Cohort 3. Under this setting, Model 1 and Model 2 had the same type of survival function (LASSO-CPH), and hence, the differences in the input data would explain the differences in performance. On the other hand, Model 2 had the same input data as our proposed CNN-Survival (Model 3) as they both used features from the dense layer. Given that, the performance disparities of Model 2 and Model 3 can be explained by the different survival functions where Model 2 uses LASSO-CPH and instead, Model 3 uses the modified loss function to generate survival probabilities for a given time. The performance of all three models was validated in Cohort 3 (test set) at 18 months by concordance index (CI) and index of prediction accuracy (IPA) using R software (version 3.5.3), Survival, Survcomp, and riskRegression library [39‐41].