Spatial transcriptomics and artificial intelligence: a scoping review of emerging applications in head and neck pathology

Artificial intelligence (AI) applied to whole slide imaging (WSI), coupled with spatial transcriptomics (ST) and proteomics, has the potential to not only standardize diagnosis but also uncover microenvironmental signatures that drive disease progression and treatment resistance. Therefore, the field of head and neck (H&N) pathology can substantially benefit from the application of these new technologies especially to precursor lesions or in cases with a high degree of inter-pathologist variability and subjectivity such as dysplasia grading in Oral Potentially Malignant Disorders (OPMDs).

OPMDs include oral mucosal abnormalities that are associated with a statistically increased risk of developing oral cancer [1]. Despite continuous clinical monitoring of OPMDs by clinicians, the precise timing and underlying mechanisms governing malignant transformation remain poorly understood. Furthermore, clinical and histopathological evaluation of these lesions can be challenging due to overlapping features with lichenoid lesions, potentially complicating definitive diagnosis. Clinically, distinguishing true OPMDs from benign reactive, infectious or inflammatory entities requires substantial clinical experience and familiarity with the diverse presentations of these lesions [1]. Correlating the clinical with the histopathological features provides an ideal opportunity for pathologists to accurately diagnose OPMDs [2]. Clinically, oral leukoplakia presents in two forms: homogeneous and non-homogeneous leukoplakia. Histologically, all forms of leukoplakia have the potential to exhibit varying grades of oral epithelial dysplasia or even early squamous cell carcinoma (SCC) [1]. Proliferative Verrucous Leukoplakia or proliferative leukoplakia harbors the highest malignant transformation rate among all OPMDs. The likelihood of oral leukoplakia development within the context of different OPMDs is also variable and difficult to predict on clinical examination and can overlap with histopathological features. Another OPMD is oral lichen planus, an immune mediated condition with a reported malignant transformation rate of approximately 1–2%, although it’s etiopathogenesis remains unclear [1]. Figure 1 provides an illustration of OPMDs combining the clinical presentation with corresponding histopathological features. The host immune lymphocytic response to neoantigens in true preneoplastic lesions can mimic oral lichenoid lesions and therefore more objective automated diagnostic tools and molecular profiling data are needed to overcome these challenges, improve diagnostic precision and guide patient management.

Bild vergrößern

Prior to the advent of single-cell transcriptomic sequencing (i.e., scRNA-seq), it was inconceivable that we could generate the expression profiles of thousands of gene targets and cells within a tissue sample [3‐5]. Despite the promise of this technology, a notable limitation is that it dissociates cells and loses spatial context, making it difficult to confidently assign gene expression to specific cell types within intact tissue architecture [3, 6]. In contrast to scRNA-seq, ST is an emerging technology that enables measurement of gene expression while preserving the spatial organization of cells within tissue sections, allowing researchers to map molecular profiles directly onto the histological architecture [7]. The identification of spatially distinct gene clusters and differentially expressed genes from ST data can facilitate the discovery of interaction niches and region-specific biomarkers that would be inaccessible from scRNA-seq alone. Spatial resolution, in particular, is critical for studying heterogeneous tissues like oral mucosa, in which distinct microenvironments, such as epithelial-stroma interfaces and immune-infiltrated zones, are expected to play pivotal roles in disease progression [8]. To further enhance diagnostic precision and overcome the inherent subjectivity in histopathological evaluation, there is increasing interest in leveraging AI especially multi-modal and interpretable models for the integrative analyses of spatial, molecular, and morphological data.

Machine learning (ML) and deep learning (DL) have emerged as powerful computational techniques capable of extracting complex patterns from high-dimensional data, particularly in biomedical contexts. Unlike traditional statistical methods that often rely on predefined hypotheses and parametric assumptions, ML/DL enable data-driven discovery by learning useful representations or features directly from data [9]. This is valuable in ST due to the high dimensionality, sparsity, and spatial autocorrelation of gene expression measurements [10]. ST analyses often involve complex workflows with AI methods employed at various steps in the data processing stack, including multi-slice data alignment [11] nuclei/cell segmentation [12‐15] and gene expression imputation based on histology [16, 17]. For example, convolutional neural networks (CNN) trained on histopathology tiles can impute spatial gene expression maps [18]. More recently, transformer-based neural network architectures have been utilized for this task [19], enabling improved risk assessment for breast cancer recurrence. Integration of ST and scRNA-seq data is another popular approach for imputing gene expression and/or determining cell type proportions [20‐22]. Clustering continues to be one of the most popular methodologies for analyzing ST data, as it can be employed instead of cell type analyses or in conjunction. Unsupervised ML methods, such as k-means or Leiden clustering [23, 24], take a reduced-dimensional gene expression matrix as input and return cluster assignments for each spatial spot. Transcriptionally distinct spatial domains can also be identified with more advanced ML methods [21, 22], including those based on graph neural networks [25]. The resulting domains or clusters are important for standard statistical analyses, like different gene expression and pathway enrichment. ML methods are also leveraged for cell–cell communication [26] and other classification or prediction tasks, requiring supervised learning methods, such as random forests or support vector machines [27]. Overall ML methods offer enhanced capacity to model nonlinearities and integrate multimodal inputs, distinguishing them from conventional statistical techniques in the analysis of ST datasets. See [28] for an entry to the literature and best practices for ST analyses.

The purpose of this scoping review is to provide an overview of the current state of ST utilization along with different AI-driven workflows for studying OPMDs and to highlight emerging opportunities for leveraging ST and AI to improve lesion classification and identification of high-risk cases, along with current limitations and open challenges.

The ten notable studies demonstrated the use of spatial transcriptomics and/or proteomics to investigate H&N cancers, particularly HPV-positive SCC, de-novo SCC and SCC associated with OPMDs. However, they leveraged ST technology in the context of different study designs and objectives. Two studies [29, 30] applied ST to corroborate immune checkpoints identified a priori and studied via immunohistochemistry (IHC). This study design had the advantage that ST, which is expensive, could be applied to a smaller number of representative samples from the larger cohort. Seven studies applied ST to characterize gene expression across tissue types and/or the tumor microenvironment (TME) more broadly. Of these studies, one [31] investigated treatment resistance with a longitudinal case study, two [32, 33] investigated malignant transformation with a cross-sectional study design, which [33] also presents a spatial atlas, and four [34‐37] investigated the role of spatial architecture in H&N SCC. The study by [35], in particular, trained an ML model on ST data to predict whether spots are at the “leading edge” of a tumor and then applied this model to different (non-H&N SCC) cancer types. The final notable study [38] was developed a cell-death-related risk score on publicly available bulk and scRNA-seq data and then generated ST for risk score validation. Overall, the ten notable studies identified gene expression signatures with prognostic and therapeutic implications, as discussed in more depth below.

Noda et al. [29] utilized imaging approach to characterize the distribution and, in some regions, the mutual exclusivity, of immune checkpoint ligands (e.g., PD-L1 and B7-H4) within H&N SCC. The study involved IHC scoring of a large cohort. Transcript–protein concordance was validated by ST sequencing (Visium) of six representative H&N SCC paired samples, with squamous intraepithelial neoplasia and normal oral mucosa. Spatial mapping of CD274 expression corroborated immunostaining, showing the strongest enrichment in invasive H&N SCC regions, identified based on histology. ST also corroborated the presence of PD-L1 mRNA and protein expression in the PD-L1–high zones identified from the IHC, supporting the biological relevance of transcript-level measurements for checkpoint stratification.

Noda et al. [30] expanded upon their original study [29], investigating B7-H4 (VTCN1) as an alternative immune checkpoint target in H&N SCC, characterized by Low PD-L1 or immune-checkpoint-inhibitor-resistance. As in the group’s first paper, they combined ST (Visium) and multiplex IHC experiments on H&N SCC samples with paired squamous intraepithelial neoplasia and normal oral mucosa. Spatial maps revealed a mutually exclusive VTCN1–CD274 pattern in 83% of cases, with CD8A down-regulated in VTCN1-positive regions, indicating immune-suppressive niches. These transcriptomic findings corroborated IHC, showing reciprocal B7-H4/PD-L1 staining across tumor cores. Clinically, the authors proposed B7-H4 as a promising antibody–drug-conjugate target for PD-L1-negative or immune-checkpoint-inhibitor-resistant H&N SCC. Unlike the other studies that aimed to broadly characterize spatial patterns of disease progression, the two investigations by Noda et al. focused on evaluating an IHC marker that had been identified a priori. Because their goal was to validate this targeted IHC finding rather than explore global spatial gene-expression heterogeneity, their analytical pipeline appropriately relied on conventional differential-expression and pathway analyses rather than ML/DL based discovery approaches.

Iwasa et al. [31] used GeoMx digital spatial profiling to study spatial changes in gene expression and TME compartments from a single patient with SCC who acquired resistance to anti-PD-1 immunotherapy (nivolumab). Tumor samples collected pre- and post- resistance, along with normal tissue. The workflow involved selecting tumor, stromal, and normal regions of interest (PanCK ±), performing whole-transcriptome profiling, and comparing samples using differential expression and Reactome pathway enrichment within the GeoMx platform. These analyses revealed that, prior to immunotherapy, the initial tumor exhibited strong expression of antigen presentation and interferon-gamma signaling genes, whereas after resistance, these pathways were suppressed and replaced by dominant epigenetic reprogramming signatures including overexpression of PRMTs, HDACs, DNMT1, and EZH2 accompanied by downregulation of MHC class I molecules. Concurrently, the TME shifted from protein synthesis-related activity to G protein-coupled receptors and olfactory receptor signaling, indicating stromal reprogramming. Overall, these results suggest that immune escape in SCC might be driven by epigenetic suppression of antigen presentation and altered stromal signaling, highlighting potential therapeutic targets to restore immunogenicity and overcome resistance, although the conclusions of this study may be limited by the single-patient, observational design and possible therapy-specific effects.

Sun et al. [33] investigated the transformation of precancerous lesions to oral SCC. scRNA-seq and ST sequencing (Visium) was applied to biopsies from nine patients, including normal mucosa, dysplastic tissue adjacent to the malignant SCC lesions, and tumoral tissues. Segmentation of these regions was performed by a pathologist based on the histological images. Transcriptionally distinct zones were defined via unsupervised clustering and marker gene identification. Clusters were used in pathway enrichment analyses. Cell–cell communication was characterized with CellphoneDB [39]. These analyses identified initiation-associated epithelial and fibroblast signatures, immune and stromal cell reprogramming (especially immune‑inhibitory monocytes and pro-tumor signaling pathways such as VEGF), all spatially localized to precancerous zones. Validation, performed via functional modeling in mice, suggests that targeting the identified pathways (e.g., anti‑PD‑1, anti‑TGFβ) may prevent malignant progression, informing future early‑intervention strategies. Lastly, the study provided an integrative single-cell and ST atlas, capturing the cellular and molecular changes underlying SCC initiation from normal mucosa to dysplastic epithelium to early tumor. While this resource offers spatial resolution across disease stages which may or may not translate to longitudinal insights into the stepwise progression of SCC, as all samples were collected at a single timepoint. This contrasts with Iwasa et al. [31], who used a longitudinal study design.

Zhi et al. [32] also explored malignant transformation with a cross-sectional design, like the study by Sun et al. [33], but focusing on oral submucous fibrosis (OSF), an entity classified as a high-risk OPMD. ST sequencing (Visium) was applied to 4 OSF-SCC transformed samples and 1 conventional SCC sample (no OSF) as a control. Regions of interest, generated via unsupervised clustering, were annotated based on marker genes. After, differential gene expression and pathway analyses were conducted. The results were overlaid with spatial metabolomics data to assign metabolic programs to tissue zones, a unique aspect of this study. This analysis enabled the delineation of inflammatory stem-like cells (ISCs) undergoing partial epithelial–mesenchymal transition (pEMT) within the in-situ carcinoma region, eventually acquiring fibroblast-like phenotypes and participating in collagen deposition. This ISC > pEMT > CAF (Cancer Associated Fibroblasts) path mirrors biological transitions suspected in high-risk OPMDs. Importantly, the enrichment of polyamine metabolism and presence of tertiary lymphoid structures suggest both diagnostic and therapeutic relevance, supporting the utility of ST in identifying metabolic vulnerabilities and immunologic context. Due to the cross-sectional nature of this study, the proposed trajectory remains inferential, with causality suggested by spatial associations rather than demonstrated through longitudinal evidence.

Shaikh et al. [34] investigated the spatial distribution and role of fibroblast and myeloid cells subtypes driving lymph node metastasis. Their study analyzed H&N SCC samples from 23 gingivo-buccal SCC patients, applying both ST and IHC. After ST profiling (GeoMx) in selected regions of interest across tumor–stromal interfaces, spatial deconvolution (unsupervised learning) was to assess cell-type composition per spot. Standard methods for differential gene expression, gene set variation, and protein–protein interaction networks were applied. Overall, the analyses identified functional organization of tumor-stromal interfaces as a critical zone for metastasis progression. These are regions within 200 μm of the tumor edge, where intermediate fibroblasts, myeloid-derived cells, and neutrophils accumulate near tumor regions, while CAF and extracellular matrix genes (e.g., FN1 and COL5A1) localize toward stromal ends, especially in lymph node–positive cases. They observed that intermediate fibroblasts transition into CAFs and migrate outward, contributing to extracellular matrix remodeling and immune suppression, suggesting this spatial reorganization and stromal activation may promote an immune-suppressive microenvironment favoring cancer invasion and lymphatic spread. This study demonstrates the importance of spatial information in identifying potential checkpoints for inhibiting cancer metastasis, in concordance with the study by Zhi et al. [32].

Liu et al. [37] investigated the interplay between metabolic heterogeneity and the tumor immune microenvironment in oral SCC [24], although unlike Zhi et al. [32], who combined spatial metabolomics with transcriptomics to map polyamine metabolism and its association with inflammatory stem-like cell transitions in OSF-derived OSCC. Instead, Liu et al. leveraged scRNA-seq and ST (Visium), along with publicly available bulk and scRNA-seq data. Data integration enabled tumor immune microenvironment identification as well as per spot metabolic activity calculations with scMetabolism. Unsupervised clustering was performed on these spatial regions, and then clusters were classified into hyper-, normal-, or hypo-metabolic niches. Lastly, cell–cell communication was evaluated in these regions to delineate the coordinate work axis of epithelia cells, inflammatory CFAs, and regulatory T cells.

Arora et al. [35] investigated spatial architecture in oral SCC, identifying a transcriptionally conserved leading-edge zone associated with tumor aggressiveness and poor prognosis. ST sequencing (Visium) was applied to samples from 10 oral SCC patients. After unsupervised clustering, marker genes were used to label clusters as “tumor core”, “leading edge”, or “transitory”. Distinct and consistent spatial architectures between tumor core and leading edge were identified via differential gene expression, functional enrichment, pathway, and regulatory analyses [note that publicly available scRNA-seq data was leveraged for cell type deconvolution, as in Liu et al. [37]. RNA velocity and kinetic modeling was used to trace cell fate trajectories in the oral SCC HPV-negative samples, leading to the identification of gene signatures and differences associated with aggressive forms of oral SCC. Lastly, the annotated ST data was used to train models for predicting whether a spot was tumor core or leading edge. The application of these models to publicly available ST data for 17 different cancer types revealed that tumor core tended to be tissue-specific but leading edge had conserved gene expression profiles across cancer types, suggesting a potential common mechanism for tumor progression. This study marks an important step toward automating ST annotation through AI, demonstrating that learned architectural patterns can be transferred across tissue types and that spatial profiling can yield predictive spatial biomarkers. This zonal dissection could be directly applicable in OPMDs to stratify regions at highest risk of malignant transformation, although the limited number of samples used to train the model may limit the robustness of such inferences, underscoring the need for large, annotated ST data sets to enable reliable model training and generalization.

Zhang et al. [36] exclusively utilized publicly available data, making it distinct from the previously discussed notable studies. ST data were taken from the previously discussed study by Arora et al. [35] and scRNA-seq were data taken from [40] and [41]. Data integration was performed with the ML method RCTD [22]. This analysis revealed associations between epithelial cell subtypes and their interaction with normal-like CAFs and tumor invasiveness. Additionally, the identification of integrin and IGF1-mediated crosstalk provided mechanistic insight into epithelial-stromal interactions driving invasiveness, in concordance with Shaikh et al. [34]. This framework could be utilized to investigate other cell types and interactions associated with tumorigenesis in OPMDs, for example peri epithelial lymphocytes in OED have been reported to be significant for malignant transformation prediction [42]. Critically, this framework can utilize existing scRNA-seq or ST data, reducing the need for data generation, which can be prohibitively expensive for ST.

Pan et al. [38] also made use of publicly available data, like Zhang et al. [36]. Specifically, they leveraged bulk and ScRNA-seq data from both HPV-related and conventional oral SCC samples to train a consensus cell death-related risk score model. This risk score was validated with ST data (Visium) generated from tissue sections of laryngeal SCC from two patients and one normal larynx control patient. ST analyses revealed that high risk scores were also associated with TGF-β signaling pathways and that the interfaces between malignant cells and immune cells were enriched for high-risk scores. Unrelated to the ST validation, high risk scores were predictive of worse overall survival. Future studies could investigate the utility of such risks to OPMDs, which enable spatially resolved modeling of cell death, for early prediction of malignant transformation. Lastly, it is worth noting that this study evaluated whether learned patterns from one data generation modality translate another, similar to Arora et al.’s [35] who attempted transfer from oral SCC to other cancer types.

To summarize, the ten notable studies summarized above differ in their scope, analytical framework, and biological focus but are united by a common theme: spatial information is crucial for exposing the spatial interplay of cell types, signaling networks, and microenvironmental states that drive malignant transformation or cancer progression in OPMDs and H&N SCC. In many studies, joint ST and scRNA-seq data, often including data from public repositories, was also critical for combating trade-offs in spatial and molecular resolution [e.g. [35‐37]]. Strong molecular signal (sequencing depth) in particular, is critical for robust inference of metabolic or immune activity. Another common theme is the growing emphasis on multi-modal data [e.g., ST and IHC [29, 30]] and multi-omics data [e.g., ST and spatial metabolomics; e.g. [32]]. Integrated data analyses provided complementary perspectives on tissue organization, revealing that the tumor–stromal interface is not a static boundary but a dynamic and interactive zone where fibroblast reprogramming, immune exclusion, and metabolic heterogeneity converge to drive malignant progression. These results underscore that understanding OPMD and H&N SCC progression requires viewing the tissue as a coordinated ecosystem rather than isolated cell populations.

Multi-omics and multi-modal studies naturally intersect with the growing use of ML and AI to further enhance the interpretive and predictive power of spatial data. The study by Arora et al. [35] illustrates how ML can extract latent spatial patterns and predict molecular phenotypes from high-dimensional ST datasets, whereas the study by Pan et al. [38] demonstrates models trained on non-spatial transcriptomic data can be used to stratify prognostic risk in ST. DL algorithms, in particular, hold promise for integrating histopathological imaging, gene expression, and clinical metadata to derive holistic, multi-modal representations of tumor biology, enabling scalable and automated insights into OPMD and H&N SCC progression.

Lastly, only one study leveraged a longitudinal design [31], and the only two studies to explicitly investigate malignant transformation from lesions [32, 33] used a cross-sectional approach. Longitudinal or temporally reconstructed spatial analyses may be needed to directly study how metabolic rewiring, stromal differentiation, and immune modulation unfold over time and to evaluate causality and directionality in disease progression. We expect this to be especially important in the context of OPMDs where malignant transformation is a slow and spatially heterogeneous process. This motivates us to present a conceptual framework for investigating malignant transformation of OPMDs.