Validation of mitotic cell quantification via microscopy and multiple whole-slide scanners

For cancer, diagnostic evaluation of histopathology tissue requires the assessment of several parameters, including size, location, the presence of stromal invasion and vascular permeation, and proliferative capacity. These factors are important because they are associated with a variety of critical clinical measures such as malignant potential and therapeutic strategies. Ki-67 quantification, performed using immunohistochemical (IHC) staining, is used as a proliferative marker [1‐6]; however, immunohistochemistry (IHC) is expensive and has limited availability in resource-constrained regions. Quantification of proliferative activity by mitotic figures also plays a vital role in predicting tumor proliferation and is often quantitated via hematoxylin and eosin (HE) staining. Guidelines regarding the assessment of tumors include mitotic cell enumeration to determine the malignant potential and prognostic value [5‐8]. However, mitotic cell detection also has limitations regarding accuracy and reproducibility [9‐12].

Since their first release, digital pathology systems (DPS) have yielded rapid breakthroughs. Several studies have reported that a primary diagnosis based on whole-slide imaging (WSI) was non-inferior to microscopic diagnosis [13‐16]. In Europe, several WSI scanners are approved (given the CE mark), and in the US, the US Food and Drug Administration (FDA) has approved the Philips IntelliSite Pathology Solution to be marketed. Additionally, the Pharmaceuticals and Medical Devices Agency in Japan has also approved the Philips system for medical use. DPS has been shown to reduce turn-around times and costs associated with pathological diagnosis [17]. These benefits promote the practical use of DPS for clinical, pathological analysis. However, few studies have reported on the agreement between the use of WSI and microscopy for analysis of histological features at the level of individual cells (e.g., mitotic figure quantification).

Furthermore, DPS enables the use of powerful image processing algorithms for histopathological analysis. Indeed, many automated histomorphologic/cytomorphologic analysis techniques have been commercialized. Additionally, the development of automated mitotic cell detection has also progressed in recent years [18‐21]. Thus, it is essential to confirm that mitotic cell detection is accurate and reliable via DPS.

The present study focuses on mitotic cell detection and aims to evaluate mitotic cell detection using WSI with multiple scanners and to determine whether mitotic cell detection using WSI is concordant with microscopy.

All 5 observers detected a total of 155 candidate mitotic cells, using both WSIs and microscopy. The counts by all observers for each observation method are shown in Table 1. Using microscopy, 29 potential candidate mitotic cells were detected by all five observers, 8 candidates by four observers, 17 candidates by three observers, 13 candidates by two observers, and 28 candidates by one observer. The remaining 60 candidates were not detected by microscopy but were detected by WSI. Of these 60 candidate mitotic figures, the truthing panel determined that four of them were true mitotic figures, and there was otherwise little-to-no consensus. Within a scanner, only one candidate was marked by three pathologists and only six candidates were marked by two pathologists. The remaining candidate mitotic figures were marked by only one pathologist (within a scanner).

For ground truth, 37 candidates were detected by more than four observers; these were considered true mitotic cells. The consensus team evaluated the other candidates, and 74 were finally confirmed as true mitotic cells. In Fig. 1, three ground truth mitotic figures are shown: 1) example 1, confirmed as mitotic cells by all observers using all observation methods, 2) example 2, confirmed by none of the observers using scanner C, and 3) example 3, confirmed by only one observer via microscopy. All Kappa coefficients of inter-observer agreement were “substantial” to “almost perfect” (Table 2). Furthermore, all intra-observer agreements were “substantial” to “almost perfect” (Table 3).

In Fig. 2, we show the within-reader Bland Altman plots comparing log-count differences from the scanners to those with the microscope. The biases observed in the log counts show that the pathologists marked fewer MFs with the scanners compared to the microscope. They marked between 16 to 36% fewer on average and 70% fewer in some cases.

To compare all detected mitotic cell candidates with ground truth, we analyzed accuracy which is the average of sensitivity and specificity. Accuracy was between 0.631 and 0.842 across all readers and modes (Table 4, Fig. 3). After averaging over readers for each detection method, we found [36] that mitosis detection accuracy of each of the three scanners, A, B, and C, was significantly less than that of the microscope.

This study is the first, to our knowledge, to use WSIs of multiple vendors and glass microscopy to evaluate mitotic cells identification. Most previous studies used only one type of scanner to compare WSI and microscopy. In this study, we hypothesized that it is possible to assess “a certain WSI scanner of a certain manufacturer” with a task-based feature study along with the conventional examination. Hence, we used four types of scanners (all having almost the same scanning capabilities) to validate WSI as “instruments” for pathological analysis. Ultimately, inter-observer agreements within all presentation modes (microscopy and all WSI sets) were “substantial” or greater, and intra-observer agreements between all observation methods displayed a similar trend. One interesting finding worth investigating with a larger study is that the pathologists found fewer mitotic figures on the scanners than on the microscope. The present results suggest that WSI is a viable “instrument” to detect mitotic cells because it was reproducible in comparison with microscopy.

Furthermore, we attempted to evaluate mitotic cell detection in detail. Although evaluation using entire slide glass is a viable method for enumerating mitotic cells, it is not suitable for microscopic imaging of individual mitotic cells because of difficulties of annotating ROIs or target cells upon microscopic observation. The eeDAP system enabled us to evaluate mitosis at the level of individual cells. Currently, only one previous study has compared WSI and microscopy [38]. Owing to this new strategy, we could determine the ground truth of mitosis starting with the candidate mitotic cells detected by all observers. We found that it is not feasible to detect ground truth with a single pathologist, using either WSI or microscopy alone, because no observer and observation method could identify all the 74 ground truth mitotic cells. These results suggest that microscopy is also an imperfect method to detect mitotic cells.

Interestingly, microscopy was measurably more accurate than 3 of the 4 WSIs. Despite its simplicity, microscopy for detecting mitotic cells provides important information regarding malignant potential and therapeutic strategies for various tumors [5‐8]. This implies that discrepancies in mitotic cell detection may affect pathologic interpretations and thus potentially patient care. One explanation for why the accuracy of the microscope was better is that the microscope allows the pathologist to focus on different z-planes. In reality, one of the WSI sets used herein was out of focus and warranted re-scanning. Observers were concerned that the re-scanned WSIs were slightly opaque, although they could determine the histological type. Although 20× scanned WSIs are reportedly viable for histological diagnosis [16], it is expected that the resolution of WSIs scanned at 20× are insufficient to observe mitotic cells. Because pathologists often evaluate images at 40× magnification to detect mitotic cells, the image is likely to be somewhat opaque when the WSIs are magnified digitally from 20× to 40×. In addition to the capacity of the scanner, inadequate maintenance of scanners probably leads to problems in focusing. Regarding scanner C, the light source contained a covering of dust, and the glass slide stage was determined to be unstable upon periodic inspection after this study. Insufficient maintenance does not allow for optimal performance of the scanner; hence, maintenance of the scanner is also essential for WSI-based diagnosis.

Z-stacking or multilayer scanning is a viable method to resolve the issue regarding z-plane focusing. Current WSI scanners can perform z-stacking or multilayer scanning, which has enabled pathologists to adjust the focus, similar to a microscope. However, considering the number of layers and the distance between each layer, it might be that z-stacking/multilayer scanning is not sufficient for observation, owing to disadvantages such as large file sizes and extended scanning time. Adjustment of the numerical aperture (NA) may prove a promising method to resolve this issue. It is needless to say that pixel resolution also has an impact on image quality. NA is also associated with the optical resolution of the microscope and WSI; a higher NA leads to better image quality. In fact, scanner D had a higher pixel resolution (0.13 μm/pixel) and higher NA (0.95) than other scanners. Hence, it is possible to resolve this issue by employing a WSI scanner with higher pixel resolution and NA.

The condition of glass slides is also an important consideration. Generally, staining intensity of the slides becomes pale upon exposure to light. In reality, the staining intensity was reduced via microscopy when we digitized WSI images using scanner D and re-digitized using scanner C; these slides were scanned after repeated digitization by other scanners and microscopic observation. Although we considered repeating HE staining after bleaching, it was not performed because it is difficult to reproduce the original color tone completely, although re-staining will subsequently be performed. The color-tone corrective function of WSI viewers might resolve this issue; however, this was not performed in this study.

These issues about mitotic figure detection using WSI also affect computational pathology. Automated evaluation of IHC and liquid-based cytological specimens are used in clinical practice, and several studies have reported the use of automated mitotic cell detection [18‐21]. Precise recognition of mitotic cells is necessary to develop accurate automated systems to detect mitotic cells. First, the reproducibility of mitotic cell enumeration by pathologists is controversial [9‐12], and the present results also reveal that microscopic mitotic cell detection by pathologists would not have encompassed entirely ground truth as defined. Successful development of an automated mitotic cell detection system should contribute to diagnosis and therapy. To convert this to reality, we should understand and recognize the specific limitation regarding mitotic cell detection using WSI, as shown in the present study.

In this study, the arbitrary decision regarding the ground truth for comparisons represented a limitation, which was a critical and problematic issue. IHC for phosphohistone H3 (pHH3) is a popular method to detect mitotic cells; furthermore, pHH3 is an essential marker to differentiate mitosis from apoptosis [39]. pHH3 reportedly correlated with patient prognosis [40] and improved inter-observer reproducibility of mitotic cell detection [41]. IHC for pHH3 appears to be an excellent tool for mitotic cell detection; however, it is not easy to evaluate IHC results owing to background noise, false positive staining, and different staining intensities due to sample condition. These issues make it difficult for pathologists to do an accurate assessment. Furthermore, IHC for pHH3 was suggested not to be a substitute for detecting mitotic cells via HE staining [42]. Hence, performing both IHC and HE staining in the same slide or serial slides of the same tissue can effectively detect mitotic cells. However, this was not performed because all slides had already been prepared before the study. If IHC is performed for HE-stained slides after bleaching, it is not possible to assess different staining methods for the same slide with the microscope. In the current study, we identified candidate mitotic figures by many pathologists using the microscope and four WSIs, and then a consensus team reviewed the candidates.

The data and analysis scripts are available online (https://​github.​com/​DIDSR/​iMRMC/​wiki/​Tabata2019_​comparingScanner​sMFcounting).

To the best of our knowledge, this study is the first to use multiple scanners and microscopy to evaluate the detection of mitotic cells at the level of individual cells. Our results suggest that certain WSI scanners are viable “instruments” to detect mitotic cells with similar reproducibility as the microscope but with potential loss of accuracy. As such, care should be taken when using WSI to detect mitotic cells in pathological diagnosis and developing an algorithm for mitotic detection as the accuracy level is slightly inferior to microscopy. Appropriate maintenance and management of both WSI scanners and histological slides can help optimize the performance of DPS. Further development and application of WSI are expected to yield further advancements in the future.