nach oben

International Journal of Computer Assisted Radiology and Surgery

Erschienen in:

21.08.2023 | Original Article

OCIF: automatically learning the optimized clinical information fusion method for computer-aided diagnosis tasks

verfasst von: Zhaoyu Hu, Leyin Li, An Sui, Guoqing Wu, Yuanyuan Wang, Zhifeng Shi, Jinhua Yu, Liang Chen, Guiguan Yang, Yuhao Sun

Erschienen in: International Journal of Computer Assisted Radiology and Surgery | Ausgabe 12/2023

Einloggen, um Zugang zu erhalten

Abstract

Purpose

In computer-aided diagnosis, the fusion of image features extracted from neural networks and clinical information is crucial to improve diagnostic accuracy. How to integrate low-dimensional clinical information (LDCF) with high-dimensional network features (HDNF) is an urgent problem to be solved. We offer a new network search framework to address this problem, which can provide optimized LDCF fusion and efficient dimensionality reduction in HDNF.

Methods

OCIF innovatively uses Gaussian process optimization to explore the search space for the number of fully connected (FC) layers, the number of neurons in each FC layer, the activation function, the dropout factor, and whether to add clinical information to each FC layer. Moreover, OCIF employs transfer learning to reduce the training parameter space and improve search efficiency. To evaluate the effectiveness of the proposed OCIF, we utilized three popular end-to-end overall survival (OS) time prediction models to predict the three classes.

Results

Our experimental results show that applying OCIF to a classical computer-aided diagnosis neural network can improve classification accuracy. Experiments on the 2020 BRATS dataset prove that OCIF achieves satisfactory performance, with an accuracy of 0.684, precision of 0.735, recall of 0.684, and F1-score of 0.675 on the OS time prediction task.

Conclusion

OCIF effectively and creatively combines clinical information and network features, leveraging both clinical information and image features to enhance the accuracy of the final diagnosis. Our experiments demonstrate that the use of OCIF can significantly improve computer-aided diagnosis accuracy, and the approach has the potential to be extended to other medical classification tasks as well.

Nur mit Berechtigung zugänglich

Naser MA, Deen MJ (2020) Brain tumor segmentation and grading of lower-grade glioma using deep learning in MRI images. Comput Biol Med. https://doi.org/10.1016/j.compbiomed.2020.103758CrossRefPubMed

Yu JH, Shi ZF, Lian YX, Li ZJ, Liu TT, Gao Y, Wang YY, Chen L, Mao Y (2017) Noninvasive IDH1 mutation estimation based on a quantitative radiomics approach for grade II glioma. Eur Radiol 27:3509–3522. https://doi.org/10.1007/s00330-016-4653-3CrossRefPubMed

Feng X, Tustison NJ, Patel SH, Meyer CH (2020) Brain tumor segmentation using an ensemble of 3D U-nets and overall survival prediction using radiomic features. Front Comput Neurosc. https://doi.org/10.3389/fncom.2020.00025CrossRef

Zhou T, Fu H, Zhang Y, Zhang C, Lu X, Shen J, Shao L (2020) M2Net: multi-modal multi-channel network for overall survival time prediction of brain tumor patients. In: Proceedings of the international conference on medical image computing and computer-assisted intervention, pp 221–231

Chato L, Kachroo P, Latifi S (2021) An automatic overall survival time prediction system for glioma brain tumor patients based on volumetric and shape features. Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries (Brainles 2020) Pt Ii 12659:352–365. https://doi.org/10.1007/978-3-030-72087-2_31CrossRef

Tang W, Zhang H, Yu P, Kang H, Zhang R (2022) MMMNA-net for overall survival time prediction of brain tumor patients. In: Proceedings of the 2022 44th annual international conference of the IEEE engineering in medicine & biology society (EMBC), pp 3805–3808

Hu Z, Yang Z, Zhao J, Zhang H, Vaios E, Lafata K, Yin F, Wang C (2022) A deep learning model with radiomics analysis integration for glioblastoma post-resection survival prediction. Med Phys 49:E364–E364

Zheng XY, Yao Z, Huang YN, Yu YY, Wang Y, Liu YB, Mao RS, Li F, Xiao Y, Wang YY et al (2021) Deep learning radiomics can predict axillary lymph node status in early-stage breast cancer. Nat Commun. https://doi.org/10.1038/s41467-021-24605-8CrossRefPubMedPubMedCentral

Ren PZ, Xiao Y, Chang XJ, Huang PY, Li ZH, Chen XJ, Wang X (2021) A comprehensive survey of neural architecture search: challenges and solutions. Acm Comput Surv. https://doi.org/10.1145/3447582CrossRef

10.

Elsken T, Metzen JH, Hutter F (2019) Neural architecture search: a survey. J Mach Learn Res 20

11.

Kwasigroch A, Grochowski M, Mikolajczyk A (2020) Neural Architecture search for skin lesion classification. Ieee Access 8:9061–9071. https://doi.org/10.1109/Access.2020.2964424CrossRef

12.

Jiang HL, Shen FH, Gao F, Han WD (2021) Learning efficient, explainable and discriminative representations for pulmonary nodules classification. Pattern Recogn. https://doi.org/10.1016/j.patcog.2021.107825CrossRef

13.

Liu Z, Wang H, Zhang S, Wang G, Qi J (2020) Nas-scam: neural architecture search-based spatial and channel joint attention module for nuclei semantic segmentation and classification. In: Proceedings of the international conference on medical image computing and computer-assisted intervention, pp 263–272

14.

Lu ZC, Sreekumar G, Goodman E, Banzhaf W, Deb K, Boddeti VN (2021) Neural architecture transfer. Ieee T Pattern Anal 43:2971–2989. https://doi.org/10.1109/Tpami.2021.3052758CrossRef

15.

Guo XQ, Yang C, Lam PL, Woo PYM, Yuan YX (2020) Domain knowledge based brain tumor segmentation and overall survival prediction. Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries (Brainles 2019) Pt Ii 11993:285–295. https://doi.org/10.1007/978-3-030-46643-5_28CrossRef

16.

Puybareau E, Tochon G, Chazalon J, Fabrizio J (2019) Segmentation of gliomas and prediction of patient overall survival: a simple and fast procedure. Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, Brainles 2018 Pt Ii 11384:199–209. https://doi.org/10.1007/978-3-030-11726-9_18CrossRef

17.

Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun Acm 60:84–90. https://doi.org/10.1145/3065386CrossRef

18.

Tayal S, George PM, Singla P, Kose U (2022) Bayesian reasoning and Gaussian processes for machine learning applications, 1st edn. Chapman & Hall/CRC Press, Boca Raton

19.

Rasmussen CE (2004) Gaussian processes in machine learning. Adv Lect Mach Learn 3176:63–71. https://doi.org/10.1007/978-3-540-28650-9_4CrossRef

20.

Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Global Optim 13:455–492. https://doi.org/10.1023/A:1008306431147CrossRef

21.

Menze BH, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby J, Burren Y, Porz N, Slotboom J, Wiest R et al (2015) The multimodal brain tumor image segmentation benchmark (BRATS). Ieee T Med Imaging 34:1993–2024. https://doi.org/10.1109/Tmi.2014.2377694CrossRef

22.

Wang ZH, Xiao XL, Zhang ZT, He K, Hu F (2021) A radiomics model for predicting early recurrence in grade II gliomas based on preoperative multiparametric magnetic resonance imaging. Front Oncol 11:684996. https://doi.org/10.3389/fonc.2021.684996CrossRefPubMedPubMedCentral

23.

Zhou F, Li TF, Li H, Zhu HT (2017) 2018, TPCNN: two-phase patch-based convolutional neural network for automatic brain tumor segmentation and survival prediction. Brainlesion Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, Brainles 10670:274–286. https://doi.org/10.1007/978-3-319-75238-9_24CrossRef

24.

Huang G, Liu Z, van der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. Proc Cvpr Ieee 2261:2269. https://doi.org/10.1109/Cvpr.2017.243CrossRef

25.

He KM, Zhang XY, Ren SQ, Sun J (2016) Deep residual learning for image recognition. Ieee Conf Comput Vis Pattern Recogn (Cvpr) 2016:770–778. https://doi.org/10.1109/Cvpr.2016.90CrossRef

26.

Rawat W, Wang ZH (2017) Deep convolutional neural networks for image classification: a comprehensive review. Neural Comput 29:2352–2449. https://doi.org/10.1162/neco_a_00990CrossRefPubMed

27.

Duchi J, Hazan E, Singer Y (2011) Adaptive subgradient methods for online learning and stochastic optimization. J Mach Learn Res 12:2121–2159

28.

Sulaiman S, Wahid RA, Ariffin AH, Zulkifli CZJTEM (2020) Question classification based on cognitive levels using linear svc. 83, 6463–6470

29.

Sharaff A, Gupta H (2019) Extra-tree classifier with metaheuristics approach for email classification. Adv Intell Syst 924:189–197. https://doi.org/10.1007/978-981-13-6861-5_17CrossRef

30.

Couronne R, Probst P, Boulesteix AL (2018) Random forest versus logistic regression: a large-scale benchmark experiment. Bmc Bioinform. https://doi.org/10.1186/s12859-018-2264-5CrossRef

31.

Garbin C, Zhu X, Marques O (2020) Dropout vs. batch normalization: an empirical study of their impact to deep learning. Multimed Tools Appl 79:12777–12815. https://doi.org/10.1007/s11042-019-08453-9CrossRef

32.

Wei X, Zhang T, Li Y, Zhang Y, Wu F (2020) Multi-modality cross attention network for image and sentence matching. In: Proceedings of the proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10941–10950

33.

Lee K-H, Chen X, Hua G, Hu H, He X (2018) Stacked cross attention for image-text matching. In: Proceedings of the Proceedings of the European conference on computer vision (ECCV), pp 201–216

Titel: OCIF: automatically learning the optimized clinical information fusion method for computer-aided diagnosis tasks
verfasst von: Zhaoyu Hu
Leyin Li
An Sui
Guoqing Wu
Yuanyuan Wang
Zhifeng Shi
Jinhua Yu
Liang Chen
Guiguan Yang
Yuhao Sun
Publikationsdatum: 21.08.2023
Verlag: Springer International Publishing
Erschienen in: International Journal of Computer Assisted Radiology and Surgery / Ausgabe 12/2023
Print ISSN: 1861-6410
Elektronische ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-023-02985-0

Update Radiologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.

Newsletter bestellen

Klimawandel und Gesundheit: Was Sie in der Hausarztpraxis wissen müssen und tun können

Springer Medizin

OCIF: automatically learning the optimized clinical information fusion method for computer-aided diagnosis tasks