Novel rapid intraoperative qualitative tumor detection by a residual convolutional neural network using label-free stimulated Raman scattering microscopy

Rapid intraoperative histopathological analysis of fresh frozen tumor tissue is an essential tool for biopsy and surgical resection control [1]. This is based on a time-consuming and costly intraoperative pathology workflow for intraoperative decision-making that often leads to prolonged surgery times with the patient kept in anesthesia waiting for the pathological diagnosis [2, 3]. In addition, not every hospital providing surgeries in patients with brain tumors holds a (neuro)pathology unit available for rapid fresh frozen diagnostics and even if, it is usually only available during normal working hours. Recently, a fully automated label-free optical laser system creating digital hematoxylin-and-eosin (H&E)-like images called stimulated Raman histology (SRH) has been introduced as an alternative to fresh frozen conventional histopathological section diagnostics [4‐8]. In brief, SRH is an optical digital molecular microscopy technique that exploits the intrinsic optical properties of biological macromolecules, such as lipids, proteins, and ribonucleic acids, and converts them into digital H&E-like images within 3–4 min. The SRH images are then to be reviewed by a neuropathologist or available for analyses of a machine learning algorithm. The combination of SRH and the use of machine-learning techniques for analysis in a semi-automated workflow has already been explored in some proof-of-concepts investigations [8, 9]. Using convolutional neural networks and the associated evaluation of SRH images, various neurosurgical tumor entities can already be reliably detected [4, 5]. This possibility provides the neurosurgeon during the surgical resection with important information about the tumor entity and glial malignancy to tailor the surgery accordingly. However, the recently introduced CNN to determine the tumor entity and malignancy in the case of gliomas is especially limited in reliably detecting tumor tissue in the tumor margins for resection extent and biopsy control. This study aimed to develop a novel SRH-coupled deep residual CNN for qualitative brain tumor detection independent of entity and dignity and test its reliability to distinguish between tumor and non-tumor brain tissue in a semi-automated clinical intraoperative workflow.

In a prospective single-center study design patients were recruited in 2021 after they were indicated for surgery or stereotactic biopsy based on decisions of an interdisciplinary neuro-oncological tumor board panel. This study was reviewed and approved by the local ethics committee (Nr. 21-1238). All patients gave their written informed consent for the scientific use of their data according to European law. Inclusion criteria comprised: (1) Suspected tumor lesions of the central nervous system, (2) aged over 18 years, and (3) the patient is willing and able to give informed consent for participation. Exclusion criteria comprised: (1) Patient does not agree to participate in the study or (2) unable to or unwilling to give informed consent. Specimen samples were excluded if the collected specimen was inadequate, e.g. broken slide or specimen sample size under 1.7 mm diameter for SRH imaging. All specimen samples obtained were imaged immediately after collection by the intraoperative label-free fiber-laser-based SRS microscope and evaluated by the CNN as the output class (1) tumor, (2) non-tumor, or (3) low quality. An independent neuropathologist reviewed all generated SRH images as a control arm.

One of the main challenges in neurosurgery remains to distinguish tumorous tissue safely and efficiently from non-tumorous brain tissue to enable successful stereotactic biopsies and surgically complete resections. In addition, reliable quick histopathological diagnostics are increasingly needed for intraoperative decision-making and gain importance, e.g. for applying local adjuvant treatment options such as chemotherapeutic wafers [11] or intraoperative radiotherapy [12, 13]. The use of conventional rapid frozen section diagnostics with tissue processing and sectioning and white light microscopy by the (neuro)pathologist is the gold standard of care but involves a physical and time-consuming disruption of the surgical workflow [3]. Despite efficiency optimization of the process over years it remains user-dependent, costly- and time-consuming with 20 to 30 min turnaround time, in some cancer centers even longer, and needs high resource requirements due to interdepartmental logistical challenges [3, 14]. Furthermore, destructive, freezing, and compressing artifacts can occur during tissue processing [14]. Lastly, a (neuro)pathology unit is not always available in every surgical tumor center.

As an alternative histopathological assessment pathway the digital SRS microscopy has recently been introduced in the clinical setting as a label- and dye-free microscopic chemical imaging technique for unprocessed tissue specimens [8, 9]. The additive and encouraging use of AI applications with SRH microscopy has already been reported, allowing rapid prediction of tumor entity and glial dignity with encouraging agreement with the neuropathologist's diagnosis [4, 5]. Hollon et al. recently demonstrated an SRH-based CNN to distinguish between predefined tumor entities and dignity [5] but was limited in a qualitative conclusion on whether tumor or no tumor tissue is present. A qualitative assessment of a CNN that distinguishes between tumor and non-tumor has not yet been reported.

We developed three CNNs trained and tested on the same dataset and examined their diagnostic performance, reliability between CNNs, between CNNs and a neuropathologist, and the IC of multiple random areas within a sample.

Using cellular and nuclear morphological features, our trained and deployed CNNs were able to accurately identify and discriminate tumor from non-tumorous brain tissue or judge them as low-quality tissue by analyzing SRH images in a fully automated pipeline.

No significant differences in terms of probability prediction values of the tumor, non-tumor, and low-quality class were found between the three trained CNNs. In this context, previous studies show similar values but did not report detailed analyses of prediction probability values of several CNN models [4, 5, 14, 15]. We were able to demonstrate excellent concordance of the diagnosis prediction between the three CNNs, although some differences in qualitative image inspection were found at the patch size level, which did not negatively affect the overall prediction output of the whole SRH image.

Furthermore, in the reliability analysis between the CNNs and the independent neuropathological evaluation as the gold standard, the third developed CNN demonstrated the highest inter-rater agreement with approximately 90% and showed excellent diagnostic sensitivity and specificity regarding the evaluation of tumor, non-tumor tissue, and low-quality SRH images. It was therefore chosen for further validation and analysis. Other studies showed similar overall accuracy as well as ROC-AUC ranging from approximately 84 to 97% between the CNN- and neuropathological-based SRH image analysis [4, 5, 15, 16]. However, it should be noted that the CNN deployed in these studies had different output classes with different loss functions.

The question has been previously raised whether multiple scans in different areas of the same sample may even increase the diagnostic sensitivity. However, the intra-sample consistency throughout multiple areas has not yet been addressed in previous studies. In that regard, we found an excellent consistency of the applied prediction output between the three different random areas from the corresponding specimen slide of tumor, tumor margin area, or healthy brain tissue. Even if small differences were observed on patch size level, the renormalized summation could anyway result in a consistent diagnosis. In the evaluation of IC, the third trained CNN with the highest agreement in accordance with the neuropathological evaluation also showed an excellent agreement between three random areas within a specimen sample in 90.2%. This suggests that multiple scans within one sample in random areas may be deemed no longer necessary, resulting in an additional intraoperative time-saving.

Despite intraoperative technological advances like neuronavigation, intraoperative MRI-guidance, fluorescence-guided surgery, ultrasound modalities, and recently augmented reality applications [17‐23], the surgeon still relies on rapid intraoperative histopathological diagnoses on a qualitative microscopic level for decision-making, as the aforementioned techniques have diverse limitations such as being prone to potential brain shift, limited resolution sensitivity and signal specificity as well as dependent to the varying uptake of fluorescent dyes used [18, 24].

Alternative advanced techniques to our presented AI-based intraoperative optical laser system for qualitative diagnostics are the intraoperative confocal laser endomicroscopy. It provides noninvasive real-time imaging. However, this technique is limited by the need for potential toxic fluorophores for contrast-enhancement and microstructure visualization [25]. Moreover, this technique requires intense user training and merely provides non-static gray scale confocal microscopic images [26‐28], which is challenging imaging data to interpret even for trained neuropathologists. In comparison, the CNN-based label-free SRH image acquisition and analysis studied here provides user independently fast feedback, and applicability in the operating room ex-situ and digital images familiarly color-coded like conventional H&E staining. The immediate assessment and connectivity through digital image transfer systems may even provide rapid image and analysis review by the (neuro)pathologist remotely.

Further randomized prospective studies in comparison to conventional histopathological workflows are mandatory for more detailed analysis and validation to establish such a workflow as a potential standard of care on a sustainable basis.

A limitation of our study is that the neuropathological evaluation and, thus, assessment of the ground truth for CNN comparison was performed by only one board-certified neuropathologist. However, all pathological reviews of the SRH images have been done by the same trained independent neuropathology attending with over 20 years of experience minimizing the systematic error. In addition, the diagnosis strategies differed between the CNNs (highest probability prediction value) and the neuropathologist (binary value), which could influence the final comparison and therefore the accuracy of comparison between both. Since the CNN algorithm sums the patches and averages their prediction values for each of the three output classes, rather than providing the highest tumor prediction value of any of the patches, the sensitivity of CNN-based tumor probability diagnosis is, thus, limited in images containing very small tumor infiltrations. That is, in particular, true in cases when single or very few patches predict tumor—equivalent to tumor infiltration in the low micrometer range—while the rest of the SRH image is free of tumor. However, the averaging approach minimizes the misdiagnosis of the sample due to potential artifactual tumor detections in individual patches given the small size of a single patch (14.2 × 13.8 µm) relative to the whole SRH image (1.7 × 1.8 mm). In order to still enable the image reader’s correct interpretation of the CNN results in those cases, SRH image overlaying probability maps are provided in addition and spatially color-code these small tumor areas for a focused look where the CNN has detected tumor.

Additionally, it reports on a large dataset representing the entire spectrum of neurooncological diagnoses.

In conclusion, CNN-based intraoperative evaluation of SRH images enables label-free tumor detection, irrespective of dignity and entity, in a fully automated pipeline and provides a reproducible alternative pathway for intraoperative decision-making in neurooncological surgery, independent of the conventional histopathologic frozen section examination.