Skip to main content
Erschienen in:

16.10.2024 | Abdominal Radiology

One novel transfer learning-based CLIP model combined with self-attention mechanism for differentiating the tumor-stroma ratio in pancreatic ductal adenocarcinoma

verfasst von: Hongfan Liao, Jiang Yuan, Chunhua Liu, Jiao Zhang, Yaying Yang, Hongwei Liang, Haotian Liu, Shanxiong Chen, Yongmei Li

Erschienen in: La radiologia medica | Ausgabe 11/2024

Einloggen, um Zugang zu erhalten

Abstract

Purpose

To develop a contrastive language-image pretraining (CLIP) model based on transfer learning and combined with self-attention mechanism to predict the tumor-stroma ratio (TSR) in pancreatic ductal adenocarcinoma on preoperative enhanced CT images, in order to understand the biological characteristics of tumors for risk stratification and guiding feature fusion during artificial intelligence-based model representation.

Material and methods

This retrospective study collected a total of 207 PDAC patients from three hospitals. TSR assessments were performed on surgical specimens by pathologists and divided into high TSR and low TSR groups. This study developed one novel CLIP-adapter model that integrates the CLIP paradigm with a self-attention mechanism for better utilizing features from multi-phase imaging, thereby enhancing the accuracy and reliability of tumor-stroma ratio predictions. Additionally, clinical variables, traditional radiomics model and deep learning models (ResNet50, ResNet101, ViT_Base_32, ViT_Base_16) were constructed for comparison.

Results

The models showed significant efficacy in predicting TSR in PDAC. The performance of the CLIP-adapter model based on multi-phase feature fusion was superior to that based on any single phase (arterial or venous phase). The CLIP-adapter model outperformed traditional radiomics models and deep learning models, with CLIP-adapter_ViT_Base_32 performing the best, achieving the highest AUC (0.978) and accuracy (0.921) in the test set. Kaplan–Meier survival analysis showed longer overall survival in patients with low TSR compared to those with high TSR.

Conclusion

The CLIP-adapter model designed in this study provides a safe and accurate method for predicting the TSR in PDAC. The feature fusion module based on multi-modal (image and text) and multi-phase (arterial and venous phase) significantly improves model performance.
Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Zurück zum Zitat Park W, Chawla A, O’Reilly EM (2021) Pancreatic cancer: a review. JAMA-J Am Med Assoc 326(9):851–862CrossRef Park W, Chawla A, O’Reilly EM (2021) Pancreatic cancer: a review. JAMA-J Am Med Assoc 326(9):851–862CrossRef
2.
Zurück zum Zitat Klein AP (2021) Pancreatic cancer epidemiology: understanding the role of lifestyle and inherited risk factors. Nat Rev Gastroenterol Hepatol 18(7):493–502CrossRefPubMedPubMedCentral Klein AP (2021) Pancreatic cancer epidemiology: understanding the role of lifestyle and inherited risk factors. Nat Rev Gastroenterol Hepatol 18(7):493–502CrossRefPubMedPubMedCentral
3.
Zurück zum Zitat Rahib L, Smith BD, Aizenberg R et al (2014) Projecting cancer incidence and deaths to 2030: the unexpected burden of thyroid, liver, and pancreas cancers in the United States. Cancer Res 74(11):2913–2921CrossRefPubMed Rahib L, Smith BD, Aizenberg R et al (2014) Projecting cancer incidence and deaths to 2030: the unexpected burden of thyroid, liver, and pancreas cancers in the United States. Cancer Res 74(11):2913–2921CrossRefPubMed
4.
Zurück zum Zitat Hartwig W, Werner J, Jäger D et al (2013) Improvement of surgical results for pancreatic cancer. Lancet Oncol 14(11):e476–e485CrossRefPubMed Hartwig W, Werner J, Jäger D et al (2013) Improvement of surgical results for pancreatic cancer. Lancet Oncol 14(11):e476–e485CrossRefPubMed
5.
Zurück zum Zitat Khorana AA, Mangu PB, Berlin J et al (2017) Potentially curable pancreatic cancer: American society of clinical oncology clinical practice guideline update. J Clin Oncol 35(20):2324–2328CrossRefPubMed Khorana AA, Mangu PB, Berlin J et al (2017) Potentially curable pancreatic cancer: American society of clinical oncology clinical practice guideline update. J Clin Oncol 35(20):2324–2328CrossRefPubMed
6.
Zurück zum Zitat Xie T, Wang X, Li M et al (2020) Pancreatic ductal adenocarcinoma: a radiomics nomogram outperforms clinical model and TNM staging for survival estimation after curative resection. Eur Radiol 30(5):2513–2524CrossRefPubMed Xie T, Wang X, Li M et al (2020) Pancreatic ductal adenocarcinoma: a radiomics nomogram outperforms clinical model and TNM staging for survival estimation after curative resection. Eur Radiol 30(5):2513–2524CrossRefPubMed
7.
Zurück zum Zitat Birnbaum DJ, Bertucci F, Finetti P et al (2019) Head and body/tail pancreatic carcinomas are not the same tumors. Cancers (Basel) 11(4):497CrossRefPubMed Birnbaum DJ, Bertucci F, Finetti P et al (2019) Head and body/tail pancreatic carcinomas are not the same tumors. Cancers (Basel) 11(4):497CrossRefPubMed
8.
Zurück zum Zitat Sherman MH, Beatty GL (2023) Tumor microenvironment in pancreatic cancer pathogenesis and therapeutic resistance. Annu Rev Pathol 18:123–148CrossRefPubMed Sherman MH, Beatty GL (2023) Tumor microenvironment in pancreatic cancer pathogenesis and therapeutic resistance. Annu Rev Pathol 18:123–148CrossRefPubMed
9.
Zurück zum Zitat Khalaf K, Hana D, Chou JT et al (2021) Aspects of the Tumor Microenvironment Involved in Immune Resistance and Drug Resistance. Front Immunol 12:656364CrossRefPubMedPubMedCentral Khalaf K, Hana D, Chou JT et al (2021) Aspects of the Tumor Microenvironment Involved in Immune Resistance and Drug Resistance. Front Immunol 12:656364CrossRefPubMedPubMedCentral
10.
Zurück zum Zitat Chitty JL, Yam M, Perryman L et al (2023) A first-in-class pan-lysyl oxidase inhibitor impairs stromal remodeling and enhances gemcitabine response and survival in pancreatic cancer. Nat Cancer 4(9):1326–1344CrossRefPubMedPubMedCentral Chitty JL, Yam M, Perryman L et al (2023) A first-in-class pan-lysyl oxidase inhibitor impairs stromal remodeling and enhances gemcitabine response and survival in pancreatic cancer. Nat Cancer 4(9):1326–1344CrossRefPubMedPubMedCentral
11.
Zurück zum Zitat Almangush A, Jouhi L, Haglund C et al (2023) Tumor-stroma ratio is a promising prognostic classifier in oropharyngeal cancer. Hum Pathol 136:16–24CrossRefPubMed Almangush A, Jouhi L, Haglund C et al (2023) Tumor-stroma ratio is a promising prognostic classifier in oropharyngeal cancer. Hum Pathol 136:16–24CrossRefPubMed
12.
Zurück zum Zitat Uzun MA, Tilki M, Gönültaş A et al (2022) Is the tumor-stroma ratio a prognostic factor in gallbladder cancer? Rev Assoc Med Bras 68(5):664–669CrossRefPubMed Uzun MA, Tilki M, Gönültaş A et al (2022) Is the tumor-stroma ratio a prognostic factor in gallbladder cancer? Rev Assoc Med Bras 68(5):664–669CrossRefPubMed
13.
Zurück zum Zitat Han X, Xu Y, Geranpayehvaghei M et al (2020) Emerging nanomedicines for anti-stromal therapy against desmoplastic tumors. Biomaterials 232:119745CrossRefPubMed Han X, Xu Y, Geranpayehvaghei M et al (2020) Emerging nanomedicines for anti-stromal therapy against desmoplastic tumors. Biomaterials 232:119745CrossRefPubMed
14.
15.
Zurück zum Zitat Mayer P, Jiang Y, Kuder TA et al (2020) Diffusion kurtosis imaging-a superior approach to assess tumor-stroma ratio in pancreatic ductal adenocarcinoma. Cancers (Basel) 12(6):1656CrossRefPubMedPubMedCentral Mayer P, Jiang Y, Kuder TA et al (2020) Diffusion kurtosis imaging-a superior approach to assess tumor-stroma ratio in pancreatic ductal adenocarcinoma. Cancers (Basel) 12(6):1656CrossRefPubMedPubMedCentral
16.
Zurück zum Zitat Cai X, Gao F, Qi Y et al (2020) Pancreatic adenocarcinoma: quantitative CT features are correlated with fbrous stromal fraction and help predict outcome after resection. Eur Radiol 30(9):5158–5169CrossRefPubMed Cai X, Gao F, Qi Y et al (2020) Pancreatic adenocarcinoma: quantitative CT features are correlated with fbrous stromal fraction and help predict outcome after resection. Eur Radiol 30(9):5158–5169CrossRefPubMed
17.
Zurück zum Zitat Wu YJ, Wu FZ, Yang SC et al (2022) Radiomics in early lung cancer diagnosis: from diagnosis to clinical decision support and education. Diagnostics (Basel) 12(5):1064CrossRefPubMed Wu YJ, Wu FZ, Yang SC et al (2022) Radiomics in early lung cancer diagnosis: from diagnosis to clinical decision support and education. Diagnostics (Basel) 12(5):1064CrossRefPubMed
18.
Zurück zum Zitat Liao H, Yang J, Li Y et al (2022) One 3D VOI-based deep learning radiomics strategy, clinical model and radiologists for predicting lymph node metastases in pancreatic ductal adenocarcinoma based on multiphasic contrast-enhanced computer tomography. Front Oncol 12:990156CrossRefPubMedPubMedCentral Liao H, Yang J, Li Y et al (2022) One 3D VOI-based deep learning radiomics strategy, clinical model and radiologists for predicting lymph node metastases in pancreatic ductal adenocarcinoma based on multiphasic contrast-enhanced computer tomography. Front Oncol 12:990156CrossRefPubMedPubMedCentral
19.
Zurück zum Zitat Liao H, Li Y, Yang Y et al (2022) Comparison of multiple radiomics models for identifying histological grade of pancreatic ductal adenocarcinoma preoperatively based on multiphasic contrast-enhanced computed tomography: a two-center study in Southwest China. Diagnostics (Basel) 12(8):1915CrossRefPubMed Liao H, Li Y, Yang Y et al (2022) Comparison of multiple radiomics models for identifying histological grade of pancreatic ductal adenocarcinoma preoperatively based on multiphasic contrast-enhanced computed tomography: a two-center study in Southwest China. Diagnostics (Basel) 12(8):1915CrossRefPubMed
20.
Zurück zum Zitat Yokoyama S, Hamada T, Higashi M et al (2020) Predicted prognosis of patients with pancreatic cancer by machine learning. Clin Cancer Res 26(10):2411–2421CrossRefPubMed Yokoyama S, Hamada T, Higashi M et al (2020) Predicted prognosis of patients with pancreatic cancer by machine learning. Clin Cancer Res 26(10):2411–2421CrossRefPubMed
21.
Zurück zum Zitat Toseef M, Olayemi Petinrin O et al (2023) Deep transfer learning for clinical decision-making based on high-throughput data: comprehensive survey with benchmark results. Brief Bioinform 24(4):bbad254CrossRefPubMed Toseef M, Olayemi Petinrin O et al (2023) Deep transfer learning for clinical decision-making based on high-throughput data: comprehensive survey with benchmark results. Brief Bioinform 24(4):bbad254CrossRefPubMed
22.
Zurück zum Zitat Fabijan A, Fabijan R, Zawadzka-Fabijan A et al (2023) Evaluating scoliosis severity based on posturographic X-ray images using a contrastive language-image pretraining model. Diagnostics (Basel) 13(13):2142CrossRefPubMed Fabijan A, Fabijan R, Zawadzka-Fabijan A et al (2023) Evaluating scoliosis severity based on posturographic X-ray images using a contrastive language-image pretraining model. Diagnostics (Basel) 13(13):2142CrossRefPubMed
23.
Zurück zum Zitat Collins GS, Reitsma JB, Altman DG et al (2015) Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ 350:g7594CrossRefPubMed Collins GS, Reitsma JB, Altman DG et al (2015) Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ 350:g7594CrossRefPubMed
24.
Zurück zum Zitat Nagtegaal I, Odze R, Klimstra D (2020) The 2019 WHO classification of tumours of the digestive system. Histopathology 76(2):182–188CrossRefPubMed Nagtegaal I, Odze R, Klimstra D (2020) The 2019 WHO classification of tumours of the digestive system. Histopathology 76(2):182–188CrossRefPubMed
25.
Zurück zum Zitat Meng Y, Zhang H, Li Q et al (2021) Magnetic resonance radiomics and machine-learning models: an approach for evaluating tumor-stroma ratio in patients with pancreatic ductal adenocarcinoma. Acad Radiol 29(4):523–535CrossRefPubMed Meng Y, Zhang H, Li Q et al (2021) Magnetic resonance radiomics and machine-learning models: an approach for evaluating tumor-stroma ratio in patients with pancreatic ductal adenocarcinoma. Acad Radiol 29(4):523–535CrossRefPubMed
26.
Zurück zum Zitat Meng Y, Zhang H, Li Q et al (2021) CT Radiomics and machine-learning models for predicting tumor-stroma ratio in patients with pancreatic ductal adenocarcinoma. Front Oncol 11:707288CrossRefPubMedPubMedCentral Meng Y, Zhang H, Li Q et al (2021) CT Radiomics and machine-learning models for predicting tumor-stroma ratio in patients with pancreatic ductal adenocarcinoma. Front Oncol 11:707288CrossRefPubMedPubMedCentral
27.
Zurück zum Zitat Mongan J, Moy L, Kahn CE Jr (2020) Checklist for artificial intelligence in medical imaging (CLAIM): a guide for authors and reviewers. Radiol Artif Intell. 25(2):e200029CrossRef Mongan J, Moy L, Kahn CE Jr (2020) Checklist for artificial intelligence in medical imaging (CLAIM): a guide for authors and reviewers. Radiol Artif Intell. 25(2):e200029CrossRef
28.
Zurück zum Zitat Kocak B, Chepelev LL, Chu LC et al (2023) Assessment of RadiomIcS rEsearch (ARISE): a brief guide for authors, reviewers, and readers from the Scientific Editorial Board of European Radiology. Eur Radiol 33(11):7556–7560CrossRefPubMed Kocak B, Chepelev LL, Chu LC et al (2023) Assessment of RadiomIcS rEsearch (ARISE): a brief guide for authors, reviewers, and readers from the Scientific Editorial Board of European Radiology. Eur Radiol 33(11):7556–7560CrossRefPubMed
29.
Zurück zum Zitat Zwanenburg A, Vallières M, Abdalah MA et al (2020) The image biomarker standardization initiative: standardized quantitative radiomics for high-throughput image-based phenotyping. Radiology 295(2):328–338CrossRefPubMed Zwanenburg A, Vallières M, Abdalah MA et al (2020) The image biomarker standardization initiative: standardized quantitative radiomics for high-throughput image-based phenotyping. Radiology 295(2):328–338CrossRefPubMed
30.
31.
Zurück zum Zitat Soffer S, Ben-Cohen A, Shimon O et al (2019) Convolutional neural networks for radiologic images: a radiologist’s guide. Radiology 290(3):590–606CrossRefPubMed Soffer S, Ben-Cohen A, Shimon O et al (2019) Convolutional neural networks for radiologic images: a radiologist’s guide. Radiology 290(3):590–606CrossRefPubMed
32.
Zurück zum Zitat Sun S, Yue X, Zhao H et al (2023) Patch-based separable transformer for visual recognition. IEEE Trans Pattern Anal Mach Intell 45(7):9241–9247PubMed Sun S, Yue X, Zhao H et al (2023) Patch-based separable transformer for visual recognition. IEEE Trans Pattern Anal Mach Intell 45(7):9241–9247PubMed
34.
Zurück zum Zitat Bever KM, Sugar EA, Bigelow E et al (2015) The prognostic value of stroma in pancreatic cancer in patients receiving adjuvant therapy. HPB (Oxford) 17(4):292–298CrossRefPubMed Bever KM, Sugar EA, Bigelow E et al (2015) The prognostic value of stroma in pancreatic cancer in patients receiving adjuvant therapy. HPB (Oxford) 17(4):292–298CrossRefPubMed
35.
Zurück zum Zitat Maloney E, DuFort CC, Provenzano PP et al (2019) Non-invasive monitoring of stromal biophysics with targeted depletion of hyaluronan in pancreatic ductal adenocarcinoma. Cancers (Basel). 11(6):772CrossRefPubMedPubMedCentral Maloney E, DuFort CC, Provenzano PP et al (2019) Non-invasive monitoring of stromal biophysics with targeted depletion of hyaluronan in pancreatic ductal adenocarcinoma. Cancers (Basel). 11(6):772CrossRefPubMedPubMedCentral
36.
Zurück zum Zitat Heid I, Steiger K, Trajkovic-Arsic M et al (2017) Co-clinical assessment of tumor cellularity in pancreatic cancer. Clin Cancer Res 23(6):1461–1470CrossRefPubMed Heid I, Steiger K, Trajkovic-Arsic M et al (2017) Co-clinical assessment of tumor cellularity in pancreatic cancer. Clin Cancer Res 23(6):1461–1470CrossRefPubMed
37.
38.
Zurück zum Zitat Chitalia RD, Kontos D (2019) Role of texture analysis in breast MRI as a cancer biomarker: a review. J Magn Reson Imaging 49(4):927–938CrossRefPubMed Chitalia RD, Kontos D (2019) Role of texture analysis in breast MRI as a cancer biomarker: a review. J Magn Reson Imaging 49(4):927–938CrossRefPubMed
39.
Zurück zum Zitat Liao H, Yuan J, Liu C et al (2023) Feasibility and effectiveness of automatic deep learning network and radiomics models for differentiating tumor stroma ratio in pancreatic ductal adenocarcinoma. Insights Imaging 14(1):223CrossRefPubMedPubMedCentral Liao H, Yuan J, Liu C et al (2023) Feasibility and effectiveness of automatic deep learning network and radiomics models for differentiating tumor stroma ratio in pancreatic ductal adenocarcinoma. Insights Imaging 14(1):223CrossRefPubMedPubMedCentral
40.
Zurück zum Zitat Yan S, Dong N, Zhang L et al (2023) CLIP-driven fine-grained text-image person re-identification. IEEE Trans Image Process 32:6032–6046CrossRefPubMed Yan S, Dong N, Zhang L et al (2023) CLIP-driven fine-grained text-image person re-identification. IEEE Trans Image Process 32:6032–6046CrossRefPubMed
41.
Zurück zum Zitat An C, Li D, Li S et al (2022) Deep learning radiomics of dual-energy computed tomography for predicting lymph node metastases of pancreatic ductal adenocarcinoma. Eur J Nucl Med Mol Imaging 49(4):1187–1199CrossRefPubMed An C, Li D, Li S et al (2022) Deep learning radiomics of dual-energy computed tomography for predicting lymph node metastases of pancreatic ductal adenocarcinoma. Eur J Nucl Med Mol Imaging 49(4):1187–1199CrossRefPubMed
Metadaten
Titel
One novel transfer learning-based CLIP model combined with self-attention mechanism for differentiating the tumor-stroma ratio in pancreatic ductal adenocarcinoma
verfasst von
Hongfan Liao
Jiang Yuan
Chunhua Liu
Jiao Zhang
Yaying Yang
Hongwei Liang
Haotian Liu
Shanxiong Chen
Yongmei Li
Publikationsdatum
16.10.2024
Verlag
Springer Milan
Erschienen in
La radiologia medica / Ausgabe 11/2024
Print ISSN: 0033-8362
Elektronische ISSN: 1826-6983
DOI
https://doi.org/10.1007/s11547-024-01902-y

Neu im Fachgebiet Radiologie

Ab sofort gelten die neuen Verordnungsausnahmen für Lipidsenker

Freie Fahrt für Lipidsenker? Das nicht, doch mit niedrigerem Schwellenwert fürs Infarktrisiko und neuen Indikationen hat der G-BA die Verordnungs-Handbremse ein gutes Stück weit gelockert.

Abdominale CT bei Kindern: 40% mit Zufallsbefunden

Wird bei Kindern mit stumpfem Trauma eine CT des Bauchraums veranlasst, sind in rund 40% der Fälle Auffälligkeiten zu sehen, die nichts mit dem Trauma zu tun haben. Die allerwenigsten davon sind klinisch relevant.

Genügt die biparametrische MRT für die Prostatadiagnostik?

Die multiparametrische Magnetresonanztomografie hat einen festen Platz im Screening auf klinisch signifikante Prostatakarzinome. Ob auch ein biparametrisches Vorgehen ausreicht, ist in einer Metaanalyse untersucht worden.

Höhere Trefferquoten bei Brustkrebsscreening dank KI?

Künstliche Intelligenz unterstützt bei der Auswertung von Mammografie-Screenings und senkt somit den Arbeitsaufwand für Radiologen. Wie wirken sich diese Technologien auf die Trefferquote und die Falsch-positiv-Rate aus? Das hat jetzt eine Studie aus Schweden untersucht.

Update Radiologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.