Serum cytokine biomarker panels for discriminating pancreatic cancer from benign pancreatic disease

Filtering of the entire dataset showed that serum levels of nine cytokines, comprising PDGF, IL-1b, IL-1ra, IL-6, IL-8, Eotaxin, IP-10, MCP-1 and MIP-1b, were significantly different between PDAC in comparisons with one or more of the control variants (Table 1). Serum levels of five cytokines, IL-1ra, IL-6, IL-8, IP-10 and MIP-1b, as well as serum CA19-9 levels were significantly increased in PDAC compared to HCs. Of these, CA19-9, IL-8 and IP-10, were also significantly elevated in PDAC compared to patients with CP, whilst a comparison of PDAC versus BBO revealed significant increases in serum levels of Eotaxin, IL-1b, MIP-1b and PDGF (Table 1). Serum levels of CA19-9, IL-8, IP-10, MIP-1b and PDGF, were significantly elevated in patients with PDAC compared to patients with benign disease (Table 1). Comparison of serum cytokine levels in subjects with obstructive jaundice showed that IL-8, IP-10, MIP-1b, PDGF and CA19-9 were all significantly elevated in PDAC compared to controls (Table 2). The circulating median levels of cytokines and CA19-9 (i.e. un-normalised) are shown in Additional file 1: Table S1. Spearman’s Rank analysis of the cytokines incorporated into panels and CA19-9 for each group showed a maximum Rho of 0.361, indicating no correlation between age and analyte level.

In the training dataset (84 PDAC, 29 HCs), serum CA19-9 had a very high performance in distinguishing patients with PDAC from healthy subjects (AUC = 0.925, CI = 0.876-0.974). Serum levels of IL-8 and IL-1b were also both found with high frequency in top ranked models of the training set in successive resamples (100% and 95% respectively) as well as CA19-9 (99%) (Figure 1A). The combined panel comprising IL-8, IL-1b and CA19-9 showed a statistically significant improvement in accuracy over CA19-9 alone (AUC = 0.984, CI = 0.968-1.00 vs. AUC = 0.925, CI = 0.876-0.974; p = 0.004; Figure 1B), although diagnostic improvement of the panel over CA19-9 was not statistically significant in the test set (PDAC = 43, HC = 16) (AUC = 0.997, CI = 0.990-1.00 vs. AUC = 0.975, CI = 0.932-1.00) (Figure 1C). Resampling the combined dataset (PDAC = 127, HC = 45) 100 times however, showed a statistically significant improved accuracy for the panel compared to CA19-9 alone (median = 94.6%, IQR = 93.8-96.4% vs. median = 89.3%, IQR = 88.4-91.1% respectively; Friedman test p < 0.001). The SN (median = 94.1%, IQR = 91.8-96.4%, versus 85.9%, IQR = 84.7-89.4% respectively) and SP (median = 100%, IQR = 96.3-100% versus median = 96.3%, IQR = 96.3-97.2%) of the panel on resampling were also significantly higher than resampled CA19-9 alone (both Friedman test p < 0.001; Figure 1D).

Model building using the training set (PDAC = 84, benign disease = 45) to distinguish PDAC patients from patients with benign pancreatic disease showed that the most frequent cytokines in top ranked resampled models were IL-8 (98%), IP-10 (76%), IL-6 (56%) and PDGF (36%) plus CA19-9 (22%) (Figure 2A). As a panel, these cytokines demonstrated improved diagnostic performance over CA19-9 alone (AUC = 0.838; CI = 0.768-0.909 vs. AUC = 0.678; CI = .579-0.776; p < 0.001) (Figure 2B) with significantly improved SN (median = 92.9%, CI = 85-97% vs. median = 53.6%, CI = 42-65%, respectively, p < 0.001), but significantly reduced SP (median = 57.8%, CI = 42-72% vs. median = 84.4%, CI = 71-94% respectively, p = 0.008). Using the independent test data (PDAC = 43, benign disease = 24), the panel showed improved performance over CA19-9 alone (AUC = 0.884, CI = 0.802-0.966 vs. AUC = 0.798, CI 0.676-0.919 respectively; p = 0.058) (Figure 2C) with a median SN of 81.4% (CI = 67-92%) and SP of 91.7% (CI = 73-99%) for the panel compared to 88.4% (CI = 75-96%) and 66.7% (CI = 45-84%) respectively for CA19-9 alone. The inconsistency in the outcomes with training and test set data here illustrate the problems associated with diagnostic performance assessment using single data splits.

Resampling of the combined dataset (PDAC = 127, benign disease n = 69) (Figure 2D) generated 100 independent optimum values for SN, SP and accuracy and showed that the accuracy of the cytokine panel (median = 77.8%, IQR = 75.6-79.4%) was significantly improved compared to CA19-9 alone (median = 67.9%, IQR = 64.1-72.5%; Friedman test p < 0.001). The SP of the cytokine panel (median = 84.8%, IQR = 80.4-89.1%) was significantly better than that of CA19-9 alone (median = 78.2%, IQR = 58.7-84.8%; Friedman test p < 0.001), while the improvement in SN was not statistically significant (median = 74.7%, IQR 70.6-77.9% vs. median = 58.8%, IQR = 52.9-80.0% respectively; Friedman test p = 0.071) (Figure 2D).

Optimum cytokine combinations for the discrimination of PDAC from CP with the training set (PDAC = 84 and CP = 32) showed that the most frequent cytokines in resampled models were IL-8 (97%), IL-6 (63%) and IP-10 (54%) as well as CA19-9 (77%) (Figure 3A). A panel comprising IL-8, IL-6 and IP-10 with CA19-9, showed a significantly improved diagnostic performance with the training data over CA19-9 alone (AUC = 0.880, CI = 0.818-0.943 vs. AUC = 0.758, CI = 0.668-0.849; p = 0.005) (Figure 3B). The SN of the panel (median = 75.0%, CI = 64-84%) was significantly better than CA19-9 alone (median = 53.6%, CI = 42-65%; p < 0.001) whilst maintaining a high SP (median = 90.6%, CI = 75-98% vs. median = 96.9%, CI = 84-100%, respectively). In the independent test set (PDAC = 43, CP = 17), the AUC achieved with the panel was greater than that of CA19-9 (0.912; CI 0.838-0.987 vs. AUC 0.848; CI 0.728-0.968, respectively) (Figure 3C).

Resampling the combined dataset (PDAC = 127 and CP = 49) revealed a significant improvement in accuracy with the cytokine panel (median = 79.5%, IQR = 77.8-82.1%) compared to CA19-9 (median = 73.5%, IQR = 64.7-76.1%; Friedman test p < 0.001) (Figure 3D). The SP was also significantly better (median = 90.9%, IQR = 87.9-93.9% vs. median = 78.8%, IQR = 69.7-93.9% respectively; Friedman test p < 0.001) with no significant difference in SN estimates (median = 76.2%, IQR = 72.6-78.6% vs. median = 73.8%, IQR = 52.3-78.6%, respectively; Friedman test p = 0.053).

The training dataset of patients with serum bilirubin levels ≥ 20 μmol/L (PDAC = 55, benign disease = 18) was resampled and the top ranked regression models most frequently featured were IP-10 (65.4%), IL-8 (50.0%), IL-1b (41.0%) and PDGF (41.0%) whilst CA19-9 was the feature of lowest importance (6.1%) (Figure 4A). The cytokine panel (IP-10,IL-8,IL-1b and PDGF) was significantly more accurate than CA19-9 (AUC = 0.810, CI = 0.693-0.927 vs. AUC = 0.614, CI = 0.445-0.782; p = 0.023) (Figure 4B) with significantly improved SN (median = 87.3%, CI = 76-95% vs. median = 63.6%, CI = 50-76% respectively; p < 0.001) and unchanged SP (median = 66.7%, CI = 41-87% vs. median = 66.7%, CI = 41-87%, respectively). The test set of samples with serum bilirubin levels ≥ 20 μmol/L (PDAC = 28, benign disease = 9) also showed diagnostic improvement (AUC = 0.857, CI = 0.699-1 vs. AUC = 0.659, CI = 0.426-0.892 respectively) (Figure 4C).

Resampling the combined dataset of patients with serum bilirubin levels ≥ 20 μmol/L (PDAC = 83 and benign disease = 27) generated 100 independent optimum values and showed significantly greater accuracy with the panel over CA19-9 (median = 73.9%, IQR = 68.5-80.8% vs. median = 63.7%, IQR = 60.3-75.3%, respectively; Friedman test p < 0.001) (Figure 4D). In particular, the SN for the cytokine panel was significantly higher than that of CA19-9 alone (median = 74.5%, IQR = 63.6-85.9% vs. median = 61.8, IQR = 54.5-85.4%, respectively; Friedman test p = 0.002).

The study included samples from both resectable and advanced PDAC cases (Additional file 1: Table S1). Binomial logistic modelling did not identify cytokines that distinguished between these two disease categories in either the training or test sets. However, post hoc tests showed that advanced and resectable PDAC were equally likely to be detected in our models as there was no significant difference in the proportion of advanced and resectable cases detected in pooled training and test set data (Pearson chi squared test for equality of proportion of detections of advanced and resectable PDAC at df =1; PDAC vs. HC, p = 0.997; PDAC vs. Benign Disease, p = 0.417; PDAC vs. CP, p = 0.704 and PDAC with obstructive jaundice vs. Benign Disease with obstructive jaundice, p = 0.892).

Blood was collected in Sarstedt Monovette Serum Z tubes (Sarstedt Ltd, Leicester, UK) and allowed to coagulate for 30 minutes before centrifugation at 800 × g for 10 min. The serum fraction was aliquotted into cryotubes and stored at -80°C. CA19-9 levels were measured using ELISA (Human Pancreatic & GI Cancer ELISA Kit, Alpha Diagnostics International, San Antonio, Tx, USA). Pre-operative total serum bilirubin (μmol/L) (Roche Modular SWA) was measured in the hospital Clinical Biochemistry Department.

The serum levels of 27 cytokines, chemokines and growth factors from patients and healthy subjects were measured blindly in duplicate using a commercially available Bio-Plex Pro 27 Plex Human Cytokine, Chemokine and Growth Factor Assay (Bio-Rad Laboratories Ltd, Hercules, CA, USA), on the Bio-Plex 200 System, with initial data analysis to measure concentration performed using Bio-Plex Manager 5.0 Software. Briefly, serially diluted standards (50 μL) and test serum, diluted 1 in 4 in sample diluent, (50 μl) was added to a microfilter plate containing antibody-coupled beads for each of the 27 analytes. The microfilter plate was incubated at room temperature on a plate shaker at 900 rpm for 1 minute followed by 300 rpm for 30 minutes. Following washing by vacuum filtration the secondary antibodies (25 μL) were added and the microfilter plate incubated as before. The microfilter plate was washed again and Streptavidin-PE (50 μL) was added and the plate incubated at room temperature on a plate shaker at 900 rpm for 1 minute followed by 300 rpm for 15 minutes. Assay buffer (125 μL) was added to each well of the microfilter plate before being read on the Bio-Plex 200 machine. Fluorescent intensities obtained for the test samples were read from the standard curve to give pg/mL values for each of the 27 cytokines, chemokines and growth factors. Ten assay plates were used to generate training data from 158 individuals, and test data from 83 individuals. To assess inter-plate variation, 6 individual samples were measured across triplicate plates; and at least one aliquot of the same PDAC patient was assayed on every plate for internal control purposes.

The training set consisted of samples from 158 subjects, 84 patients with PDAC, 45 patients with benign pancreatic disease (32 with CP and 13 with BBO due to gall stones) and 29 HCs. The serum bilirubin level of patients was recorded in all cases as either low (<20 μmol/L; upper level of normal for our Centre) or high (>20 μmol/L). In the training set there were 73 (46.2%) patients with high bilirubin, 55 with PDAC and 18 with BBO including 5 with CP. The independent test set consisted of samples from 83 subjects, 43 patients with PDAC, 17 with CP, 7 with BBO, and 16 HCs. In the test set there were 37 (44.6%) patients with high bilirubin, 28 with PDAC and 9 patients with BBO (including 2 with CP). The clinical characteristics of the training and test study populations are provided in Table 3, with further specific characteristics of cancer patients, separated into resectable and advanced categories, provided in Additional file 2: Table S2. The median age of healthy control subjects in the training set was 44 years compared to a median age of 66 years for the PDAC patients and the median age of healthy control subjects in the test set was higher at 56.5 years.

Cytokines with internal control measurements with a coefficient of variance > 50% were removed from the dataset. Cytokine concentrations (pg/mL) less than the lower limit of detection were set to 0.001. The remaining cytokines, along with CA19-9, were used in predictor model building. Normalisation between plates was undertaken by dividing raw cytokine data by the plate-specific internal control value for each cytokine. Normalised data were log₂ transformed for statistical analysis. The Shapiro-Wilk test with cut-off of p < 0.05 was applied to test the null hypothesis that data were normally distributed. Two-sample Wilcoxon signed rank (Mann–Whitney) tests with a cut-off of p < 0.05 were applied to test the null-hypothesis that the medians of groups with non-parametric (non-normal) distribution were the same. Following normalisation and log₂ transformation, the class labels (e.g. PDAC, CP, BBO or HC) were added to the data set.

Statistical modelling was performed in 64-bit R (2.15.1; R Development Core Team 2012). The training dataset was used to construct binomial logistic models, using the R function glm, to discriminate the following patient groups: PDAC versus HC; PDAC versus Benign Disease (combined CP and BBO patients) PDAC versus CP and PDAC in the presence of jaundice versus Benign Disease in the presence of jaundice.

Cytokines for inclusion in models were selected using repeated (n = 100) balanced splitting of the training set, tabulating the optimal cytokine panel for each model and then ranking cytokines. Feature finding at each data split was performed by complete enumeration of all possible logistic regression models by Morgan-Tatar search, selecting the highest ranked model as scored by Bayesian information criterion using the bestglm package in R. Cytokines were selected if they achieved a regression p < 0.05 in the top ranked model. Logistic predictor models fitted to data in this way consisted of a probability function (i.e. a predictor of class membership) of the form exp(z)/1 + exp(z) where z is a specific linear combination of coefficients, such that z = β ₀  + β ₁ x ₁  + β ₂ x ₂ …, where β _i are cytokine specific coefficients found by regression (β _o being the intercept) and x _i are sample cytokine specific values. Classification performance was assessed by Receiver Operating Characteristic (ROC) analysis using the pROC [45] and ROCR [46] packages from R to find the optimum sensitivity (SN), specificity (SP) and Area Under Curve (AUC). Comparisons of AUCs of cytokine panel models versus CA19-9 alone were performed using a one-tailed De Long’s test. The classification performance of models developed with the training data was assessed by internal validation and independently using the test set data. Confidence intervals (95%) for the SN and SP of individual diagnostic tests were calculated as exact binomial proportion intervals using the R package epiR. The McNemar test was used to compare SN and SP of paired diagnostic tests using the R package DTComPair [47].

The robustness of the classification models developed with training data was evaluated by repeatedly resampling data from the combined training and test cohorts. The combined dataset was split randomly 100 times at a ratio of 2 to 1 into resampled training and test sets, taking care to maintain the proportionate class composition of the resampled training and test sets (i.e. the ratio of cancer: control). At each split, the resampled training dataset was used to build a classification model using the cytokine panel identified during binomial logistic modelling of the original training sets. The ability of each model to classify the resampled test data was recorded. ROC analysis was performed and the SN and SP values, at the optimum accuracy, of each resampled test set were recorded. Thus, each resampled model generated 100 independent estimates of SN, SP at optimised accuracy, which were used to calculate median SN/SP and median accuracy values. Since the distribution of prediction error of resampled models was non-normal and prediction errors with the same test set but different models are not independent, the null hypothesis of equality of prediction error distribution of different models was tested using the Friedman test.