Automatic evaluation of tumor budding in immunohistochemically stained colorectal carcinomas and correlation to clinical outcome

The most commonly applied clinicopathological staging system for colorectal cancer (CRC), which is one of the most frequent solid tumors worldwide [1], is the T (tumor) N (lymph nodes) M (metastases) staging system, which classifies tumors based on primary tumor extension, regional nodal involvement and the absence or presence of metastases [2, 3].

Despite this complex and multivariate staging system, there is still room for improvement. On one hand, cases with a low T or N stage sometimes show distant metastasis, while on the other hand, high T or N stage tumors can exhibit an uneventful clinical course [4‐6]. For cases classified as intermediate according to TNM, prognostic statements are nearly impossible. It is therefore generally agreed that new features or molecular markers allowing a better stratification are necessary [6].

One morphological feature that is discussed to close this gap is the concept of tumor budding, which goes back to the 1950s, when Imai postulated the existence and biological relevance of detachment of tumor cells at the invasion front [7]. For this feature—despite the different concepts around how to define tumor budding, the problem of how to evaluate budding and the huge inter-observer variance—many studies in general, and in particular for CRC [8], have been able to show a correlation to clinical outcome and to the likelihood of nodal positivity [9, 10].

The morphological feature “tumor budding” was first described in the 1950s and showed a correlation to clinical outcome in many studies with different analytical approaches (e.g., visual assessment vs. image processing) [4, 5, 7, 9‐13], especially in colorectal carcinoma. This is notable since the definition of tumor buds is not trivial and there have been many discussions about how to assess budding.

Most research projects on tumor budding in CRC defined a tumor bud as a cluster of a few (in most studies less than 5 neighboring cells) poorly or dedifferentiated tumor cells in the desmoplastic stroma that are detached from larger tumor islands. This more or less arbitrary definition goes back to works from Gabbert et al. [14] and Hase et al. [8]. They defined tumor buds as clusters of tumor cells with a distinct morphology that could be described as epithelial-mesenchymal transition or focal dedifferentiation [14, 15]. Obviously, this has led to confusion with dedifferentiated morphology in the sense of the WHO grading and is also difficult to discriminate from a diffuse infiltration pattern [1, 8, 16].

Concerning the assessment of tumor budding, there are different approaches throughout the literature. Most approaches include focusing on hotspots, without explicitly defining them, and subsequent evaluation of the numbers of tumor buds. For instance, the German S3 guidelines from 2017, which included tumor budding as an additional risk factor for nodal positivity in early colorectal cancer, gives the following recommendations [17‐21]: 1) the invasion front should be scanned at low magnification for the area of the highest tumor budding (“hottest spot”) [17, 21]; 2) in this area, the absolute number of tumor buds should be counted [17, 21]; 3) the tumor should be graded based on the number of buds (grade 1 with 0–4 buds, grade 2 with 5–9 buds and grade 3 with < 9 buds) [17, 21].

Although tumor budding evaluation is a painstaking counting task, there are only a few works focusing on automatization. In the context of CRC, there is only one work from Caie et al. on automatic tumor budding quantification [12]. The vast majority of works rely on human evaluation with the abovementioned problems of low inter-observer correlation. Additionally, this is very time consuming and requires intensive training.

Against this background, we here establish an automatic image processing approach to reliably quantify tumor budding in immunohistochemically (IHC) stained sections of CRC samples. By publicly sharing all source codes, we hope to enable others to reproduce our results and apply it in their own scientific work or on routine histology sections. We also tested our tool on whole-slide images (WSI) with clinical annotations, investigating whether there is a correlation with clinical outcome, which has been shown for this patient cohort previously by manual counting [9, 10]. Furthermore, we also tested our approach on tissue microarrays (TMAs) to check whether this could be used for the assessment of tumor budding.

Image processing was performed in MATLAB (R2017a) on a desktop PC (Windows 7 Enterprise, Intel Core i7–4790, 32GB RAM, NVIDIA GeForce GT 630).

MATLAB-coding was carried out in accordance with style guidelines proposed by Johnson to increase the readability [25, 26]. Furthermore, object-oriented programming was applied [27], and speed up guidelines by Altman were followed [28].

In summary, all images underwent image modifying processing steps as part of the analysis, which are mentioned within the text and the legends in accordance with Digital Image Ethics [29].

To decide whether a tile (a tumor bud proposal) contained a single tumor bud or not, we used MatConvNet by Vedaldi et al. as CNN-toolbox in MATLAB [30]. For this classification task, we constructed an 8-layer CNN (see Additional file 1: Table S1). It was trained on a data set of 6292 images (100 × 100 pixel). These data set had been manually labeled by a pathologist (CAW). The dataset is available on HeiData.

The training was performed for 10,000 epochs with a constant learning rate of 10–5 on the BwUniCluster (state of Baden-Württemberg, bwHPC).

The MATLAB code of the method described below is available on GitHub (DOI: https://​doi.​org/​10.​5281/​zenodo.​1300211). It comprises tools tested for a scientific approach.

As described above, image processing was performed in the MATLAB environment. The results were saved in Excel spreadsheets (Microsoft Excel 2010, Microsoft Corporation, Redmond, WA, USA).

The clinical information and the TMA slide information (link between TMA core position and patient ID) were also saved in Excel spreadsheets. For the latter, we manually combined the automatic MATLAB generated core numbers (referred to as MATLAB core IDs) with the real-world IDs on a schematic illustration of the TMAs in a (humorously Rosetta Stone-like) Excel spreadsheet.

The information was gathered in different R databases [38]: one database for the patient level and one database for the core level. Statistical analyses were performed in R version 3.2.4 [38].

Major parts of the image processing data are also freely available on heiDATA.

To determine what sample size, how many randomly distributed cores per case were needed to reassemble the cases’ characteristics, we ran a Monte Carlo-like analysis. Therefore, (i) from every case, a predefined number of vTMAs were randomly (10 times per case and sample size) selected and then the median number of buds, the median budding score and the normalized Shannon entropy were calculated [39‐45]; (ii) then the number of random samples (n = [2:1:200]) was changed and step i was repeated.

By doing so, we obtained a range of expected values for every sample size or relative sample size (normalized to the total number of slides per case).

As previously published by Harbaum et al. [9] and Max et al. [10] on the herein analyzed patient database, there was a correlation between a high budding score based on visual estimation on whole-tissue slides to clinical parameters such as positive nodal status and inferior regression free survival. By using their data and sample set, we could independently confirm the previously published correlation between nodal status and budding score (Fig. 1a) (p < 0.05), as well as the correlation between the budding score and regression-free survival (Fig. 1b). As a new analytical feature, we could also find a correlation between the budding score and morphologic tumor grading within these datasets (Fig. 1c) [8].

There is no common generally valid definition of tumor budding in the literature. We decided to use the definition established by Satoh et al. [13], in which they defined tumor budding as “cancer cell nests of fewer than five cells in the interstitium” with subsequent grouping of budding in an interval of 5 grades (grade 0 with 0 buds, grade 1 with 1–5 buds, grade 2 with 6–10 buds, grade 3 with 10–19 buds and grade 4 with ≥20 buds). This work was chosen because of its linear grading intervals for stratification of tumor buds.

Transferred to our image processing approach, this corresponds to a stained/brown area of 72–750 μm² (300–3125 pixels) as a threshold for tumor buds (compare red circle in Fig. 2 A and histograms of the area of tumor objects in Fig. 2b). On the basis of this definition, potential tumor buds could be separated from other small tumor aggregates, which we referred to as tumor islets and which are larger in size. Since this area-based definition is prone to size variation (e.g., clusters of more than 5 very small tumor cells or stained area without nuclei) or staining variations (e.g., big structures are unequally stained leading to several stained spots in one structure), further validation steps were applied.

The detected tumor buds underwent further evaluation by cluster analysis (in regard to size, shape and border distance) and by a convolutional neural network (custom-trained MatConvNet [30]) to reduce the false positive rate. By doing so, small islets of positive staining within whole tumor mass were no longer recognized as tumor buds.

Our detection method (details in section “Tissue microarray (TMA) image processing”) was optimized and validated on 20 test cores (10 real TMA cores (rTMA) and 10 virtual TMA cores (see section “Is there a correlation between the number of tumor buds and the budding score to the nodal status in virtual TMA-cores?”)). We initially marked tumor buds manually on images of these cores (“ground truth”) and compared these findings to the automatic segmentation; regarding the absolute number of tumor buds per core, there were more discrepancies, especially for cores with high numbers of tumor buds, but still an R²-value of 0.86 was achieved. With one exception (budding score 3 instead of 2), we achieved an R²-value of 0.96 (perfect correlation) for manual vs. automatic evaluation.

In literature, tumor budding is most often defined as (i) clusters of 1–5 poorly or dedifferentiated tumor cells at the invasive front of tumors, next to larger, circumscribed tumor formations [14, 15]; (ii) and needs to be strictly discriminated from a diffuse infiltrative growth pattern [1, 8, 16]. Our image processing approach is consistent with the abovementioned definition. It takes the size and the localization in relation to the main tumor masses into account and considers the tumor bud morphology through a convolutional neural network. Furthermore, it covers the problematic area of diffuse growth or infiltration pattern by the hierarchical clustering step, which sums up such formations as one object due to the shape and localization of its subparts (step iii in section “Convolutional neural network training and application in general”.3). Several works from different groups address tumor budding for CRC and its related role as a prognostic factor, in particular in regard to nodal status [8, 10, 16, 46‐48]. Thereby, a plethora of different tumor budding assessments have been applied. For example, some groups have counted tumor buds in absolute numbers (e.g., Ueno et al. [48], Prall et al. [16]), others have stratified the absolute number into different grades (e.g., Max et al. [10]), and still others just defined high- and low-grade budding activity on the basis of the absolute numbers (Hase et al. [8]). Our approach primarily counted the number of tumor buds per high power field (one high-power field had approximately the size of a single TMA core). These numbers were then stratified into five budding grades (from grade 0 with no budding to grade 4 with extensive budding) in reference to previous works by Koelzer et al. and Satoh et al. [11, 13]. Of note, the findings of Koelzer and Satoh were mainly established based on HE staining, and their transferability to more sensitive IHC-based estimations is problematic. The higher sensitivity of the latter method could lead to higher grades [49].

In comparison to the results of a manually defined ground truth (blinded annotation by a trained pathologist, CAW), the automatic evaluation showed a very good accordance for the budding score and a good accordance for the overall number of tumor buds for whole-slide analysis (section “Tissue microarray (TMA) image processing” and Additional file 5: Figure S1), which opens the possibility of analyzing entire sections in a reliable and reproducible fashion.

For a subcohort of 20 cases, we focused on the manually delineated infiltrative border in accordance with Caie et al., who also focused on infiltrative border and showed a correlation of immunofluorescence-based image processing-based tumor budding assessment with manual budding analyses and clinical parameters [12]. Surprisingly, for our data, we could not show a correlation between the median or maximum number of tumor buds and the nodal status. Additionally, we found no correlation between budding score and nodal status. Even focusing budding analyses on hotspots (as proposed by many researchers) did not lead to a significant stratification of cases in regard to the nodal status.

By applying methods of spatial statistics [50‐53] to describe the spatial heterogeneity (as recently published for vessels in CRC [33], for lymphatic hyperplasia in the thymus [54] or for lymphatic infiltrates in the bone marrow [55]), we found significant accumulations of tumor budding foci independent of the overall frequency of tumor budding, which we called budding hotspots. The number of these budding hotspots and not their budding metrics (e.g., median budding score in the hotspots) did correlate with the nodal status. This leads obviously to the conclusion that TMAs are not suitable for analyzing tumor budding.

In addition to the nonnegligible hassle of counting tumor buds in a section, the reproducibility of human-based results and/or the training efforts to ensure such reproducibility are major limitations and hamper routine estimation of budding. Studies showed inter-observer variations in the range of kappa = 0.61–0.83 [56, 57]. However, a different human-based evaluation method using 10 high power fields (hpf) in the region with the highest density of peri-tumoral budding showed a slightly better reproducibility (kappa-values in the range 0.5–0.87) [11, 58].

Our image processing approach has been validated in terms of absolute budding number and budding score with good to very good accordance compared to manually drawn “ground truth” (Additional file 5: Figure S1). This offers the option for reproducible, time-saving whole-slide analysis and the resulting possibility of applying spatial statistics.

Interestingly, mimicking the strategies of the human evaluation (considering only the hottest spot or the 10 hottest spots) with our automatized method did not lead to significant results. Only the number of hotspots defined by spatial statistics correlated with nodal status.