nach oben

European Journal of Nuclear Medicine and Molecular Imaging

Erschienen in:

06.04.2020 | Original Article

Effect of machine learning re-sampling techniques for imbalanced datasets in ¹⁸F-FDG PET-based radiomics model on prognostication performance in cohorts of head and neck cancer patients

verfasst von: Chenyi Xie, Richard Du, Joshua WK Ho, Herbert H Pang, Keith WH Chiu, Elaine YP Lee, Varut Vardhanabhuti

Erschienen in: European Journal of Nuclear Medicine and Molecular Imaging | Ausgabe 12/2020

Einloggen, um Zugang zu erhalten

Abstract

Purpose

Biomedical data frequently contain imbalance characteristics which make achieving good predictive performance with data-driven machine learning approaches a challenging task. In this study, we investigated the impact of re-sampling techniques for imbalanced datasets in PET radiomics-based prognostication model in head and neck (HNC) cancer patients.

Methods

Radiomics analysis was performed in two cohorts of patients, including 166 patients newly diagnosed with nasopharyngeal carcinoma (NPC) in our centre and 182 HNC patients from open database. Conventional PET parameters and robust radiomics features were extracted for correlation analysis of the overall survival (OS) and disease progression-free survival (DFS). We investigated a cross-combination of 10 re-sampling methods (oversampling, undersampling, and hybrid sampling) with 4 machine learning classifiers for survival prediction. Diagnostic performance was assessed in hold-out test sets. Statistical differences were analysed using Monte Carlo cross-validations by post hoc Nemenyi analysis.

Results

Oversampling techniques like ADASYN and SMOTE could improve prediction performance in terms of G-mean and F-measures in minority class, without significant loss of F-measures in majority class. We identified optimal PET radiomics-based prediction model of OS (AUC of 0.82, G-mean of 0.77) for our NPC cohort. Similar findings that oversampling techniques improved the prediction performance were seen when this was tested on an external dataset indicating generalisability.

Conclusion

Our study showed a significant positive impact on the prediction performance in imbalanced datasets by applying re-sampling techniques. We have created an open-source solution for automated calculations and comparisons of multiple re-sampling techniques and machine learning classifiers for easy replication in future studies.

Nur mit Berechtigung zugänglich

Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2018;68(6):394–424. https://doi.org/10.3322/caac.21492.CrossRefPubMed

Gupta B, Johnson NW, Kumar N. Global epidemiology of head and neck cancers: a continuing challenge. Oncology. 2016;91(1):13–23. https://doi.org/10.1159/000446117.CrossRefPubMed

Buckler AJ, Bresolin L, Dunnick NR, Sullivan DC. A collaborative enterprise for multi-stakeholder participation in the advancement of quantitative imaging. Radiology. 2011;258(3):906–14. https://doi.org/10.1148/radiol.10100799.CrossRefPubMed

Lambin P, Rios-Velazquez E, Leijenaar R, Carvalho S, van Stiphout RG, Granton P, et al. Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer. 2012;48(4):441–6. https://doi.org/10.1016/j.ejca.2011.11.036.CrossRefPubMedPubMedCentral

Aerts HJ, Velazquez ER, Leijenaar RT, Parmar C, Grossmann P, Carvalho S, et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun. 2014;5:4006. https://doi.org/10.1038/ncomms5006.CrossRefPubMed

Cook GJ, Yip C, Siddique M, Goh V, Chicklore S, Roy A, et al. Are pretreatment 18F-FDG PET tumor textural features in non-small cell lung cancer associated with response and survival after chemoradiotherapy? J Nucl Med. 2013;54(1):19–26. https://doi.org/10.2967/jnumed.112.107375.CrossRefPubMed

He H, Garcia EA. Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 2009;21(9):1263–84.CrossRef

Kabir MF, Ludwig S, editors. Classification of breast cancer risk factors using several resampling approaches. 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA); 2018: IEEE.

Fotouhi S, Asadi S, Kattan MW. A comprehensive data level analysis for cancer diagnosis on imbalanced data. J Biomed Inform. 2019;90:103089. https://doi.org/10.1016/j.jbi.2018.12.003.CrossRefPubMed

10.

Batuwita R, Palade V, editors. Efficient resampling methods for training support vector machines with imbalanced datasets. The 2010 International Joint Conference on Neural Networks (IJCNN); 2010: IEEE.

11.

Loyola-González O, Martínez-Trinidad JF, Carrasco-Ochoa JA, García-Borroto M. Study of the impact of resampling methods for contrast pattern based classifiers in imbalanced databases. Neurocomputing. 2016;175:935–47.CrossRef

12.

Chawla NV. Data mining for imbalanced datasets: An overview. In: Data mining and knowledge discovery handbook: Springer; 2009. p. 875–86.

13.

Chawla NV, Japkowicz N, Kotcz A. Editorial: special issue on learning from imbalanced data sets. SIGKDD Explor Newsl. 2004;6(1):1–6. https://doi.org/10.1145/1007730.1007733

14.

Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP. SMOTE: synthetic minority over-sampling technique. J Artif Intell Res. 2002;16:321–57. https://doi.org/10.1613/jair.953

15.

Edge SB, Compton CC. The American Joint Committee on Cancer: the 7th edition of the AJCC cancer staging manual and the future of TNM. Ann Surg Oncol. 2010;17(6):1471–4. https://doi.org/10.1245/s10434-010-0985-4.CrossRefPubMed

16.

Vallieres M, Kay-Rivest E, Perrin LJ, Liem X, Furstoss C, Aerts H, et al. Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer. Sci Rep. 2017;7(1):10117. https://doi.org/10.1038/s41598-017-10371-5.CrossRefPubMedPubMedCentral

17.

Vallières M, Kay-Rivest E, Perrin LJ, Liem X, Furstoss C, Khaouam N et al. Data from Head-Neck-PET-CT. The Cancer Imaging Archive. The Cancer Imaging Archive; 2017. https://doi.org/10.7937/K9/TCIA.2017.8oje5q00

18.

Zhang Y, Hu J, Li J, Wang N, Li W, Zhou Y, et al. Comparison of imaging-based gross tumor volume and pathological volume determined by whole-mount serial sections in primary cervical cancer. Onco Targets Ther. 2013;6:917–23. https://doi.org/10.2147/ott.S43264.CrossRefPubMedPubMedCentral

19.

Sun H, Xin J, Zhang S, Guo Q, Lu Y, Zhai W, et al. Anatomical and functional volume concordance between FDG PET, and T2 and diffusion-weighted MRI for cervical cancer: a hybrid PET/MR study. Eur J Nucl Med Mol Imaging. 2014;41(5):898–905. https://doi.org/10.1007/s00259-013-2668-4.CrossRefPubMed

20.

Nioche C, Orlhac F, Boughdad S, Reuze S, Goya-Outi J, Robert C, et al. LIFEx: a freeware for radiomic feature calculation in multimodality imaging to accelerate advances in the characterization of tumor heterogeneity. Cancer Res. 2018;78(16):4786–9. https://doi.org/10.1158/0008-5472.Can-18-0125.CrossRefPubMed

21.

Leijenaar RT, Nalbantov G, Carvalho S, van Elmpt WJ, Troost EG, Boellaard R, et al. The effect of SUV discretization in quantitative FDG-PET radiomics: the need for standardized methodology in tumor texture analysis. Sci Rep. 2015;5:11075. https://doi.org/10.1038/srep11075.CrossRefPubMedPubMedCentral

22.

Orlhac F, Soussan M, Maisonobe JA, Garcia CA, Vanderlinden B, Buvat I, et al. Tumor texture analysis in 18F-FDG PET: relationships between texture parameters, histogram indices, standardized uptake values, metabolic volumes, and total lesion glycolysis. J Nucl Med. 2014;55(3):414–22. https://doi.org/10.2967/jnumed.113.129858.CrossRefPubMed

23.

Bailly C, Bodet-Milin C, Couespel S, Necib H, Kraeber-Bodere F, Ansquer C, et al. Revisiting the robustness of PET-based textural features in the context of multi-centric trials. PLoS One. 2016;11(7):e0159984. https://doi.org/10.1371/journal.pone.0159984.CrossRefPubMedPubMedCentral

24.

Gamer M, Lemon J, Gamer MM, Robinson A, Kendall's W. Package ‘irr’. Various coefficients of interrater reliability agreement 2012. https://cran.rproject.org/web/packages/irr/irr.pdf

25.

He H, Bai Y, Garcia EA, Li S, editors. ADASYN: Adaptive synthetic sampling approach for imbalanced learning. 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence); 2008: IEEE.

26.

Han H, Wang W-Y, Mao B-H, editors. Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning. International conference on intelligent computing; 2005: Springer.

27.

Mani I, Zhang I, editors. kNN approach to unbalanced data distributions: a case study involving information extraction. Proceedings of workshop on learning from imbalanced datasets; 2003.

28.

Tomek I. Two modifications of CNN. IEEE Trans. Syst. Man Cybern. 1976;6:769–72.

29.

Wilson DL. Asymptotic properties of nearest neighbor rules using edited data. IEEE Trans. Syst. Man Cybern. 1972;2(3):408–21.CrossRef

30.

Batista GE, Prati RC, Monard MC. A study of the behavior of several methods for balancing machine learning training data. ACM SIGKDD explorations newsletter. 2004;6(1):20–9.CrossRef

31.

Batista GE, Bazzan AL, Monard MC, editors. Balancing Training data for automated annotation of keywords: a Case Study. WOB; 2003.

32.

Alves GEDAP, Silva DF, Prati RC, editors. An experimental design to evaluate class imbalance treatment methods. 2012 11th International Conference on Machine Learning and Applications; 2012: IEEE.

33.

Chen T, Guestrin C, editors. Xgboost: a scalable tree boosting system. Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining; 2016: ACM.

34.

Powers DM. Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. J Mach Learn Technol 2011. https://doi.org/10.9735/2229-3981

35.

Demšar J. Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res. 2006;7(Jan):1–30.

36.

Lemaître G, Nogueira F, Aridas CK. Imbalanced-learn: a python toolbox to tackle the curse of imbalanced datasets in machine learning. J. Mach. Learn. Res. 2017;18(1):559–63.

37.

Lv W, Yuan Q, Wang Q, Ma J, Jiang J, Yang W, et al. Robustness versus disease differentiation when varying parameter settings in radiomics features: application to nasopharyngeal PET/CT. Eur Radiol. 2018;28(8):3245–54. https://doi.org/10.1007/s00330-018-5343-0.CrossRefPubMed

38.

Feliciani G, Fioroni F, Grassi E, Bertolini M, Rosca A, Timon G, et al. Radiomic profiling of head and neck cancer: 18F-FDG PET texture analysis as predictor of patient survival. Contrast Media Mol. Imaging. 2018. https://doi.org/10.1155/2018/3574310

39.

Peng H, Dong D, Fang MJ, Li L, Tang LL, Chen L, et al. Prognostic value of deep learning PET/CT-based radiomics: potential role for future individual induction chemotherapy in advanced nasopharyngeal carcinoma. Clin. Cancer Res. 2019;25(14):4271–9. https://doi.org/10.1158/1078-0432.Ccr-18-3065.CrossRefPubMed

40.

Zhang Y, Oikonomou A, Wong A, Haider MA, Khalvati F. Radiomics-based prognosis analysis for non-small cell lung cancer. Sci Rep. 2017;7:46349. https://doi.org/10.1038/srep46349.CrossRefPubMedPubMedCentral

41.

Park YW, Oh J, You SC, Han K, Ahn SS, Choi YS, et al. Radiomics and machine learning may accurately predict the grade and histological subtype in meningiomas using conventional and diffusion tensor imaging. Eur Radiol. 2019;29(8):4068–76. https://doi.org/10.1007/s00330-018-5830-3.CrossRefPubMed

42.

Upadhaya T, Vallières M, Chatterjee A, Lucia F, Bonaffini PA, Masson I, et al. Comparison of radiomics models built through machine learning in a multicentric context with independent testing: identical data, similar algorithms, different methodologies. IEEE Transactions on Radiation Plasma Medical Sciences. 2018;3(2):192–200.CrossRef

43.

D'Amico NC, Merone M, Sicilia R, Cordelli E, D'Antoni F, Zanetti IB, et al. Tackling imbalance radiomics in acoustic neuroma. Int. J. Data Min. Bioinform. 2019;22(4):365–88.CrossRef

44.

Gabrys HS, Buettner F, Sterzing F, Hauswald H, Bangert M. Design and selection of machine learning methods using radiomics and dosiomics for normal tissue complication probability modeling of xerostomia. Front Oncol. 2018;8:35. https://doi.org/10.3389/fonc.2018.00035.CrossRefPubMedPubMedCentral

Titel: Effect of machine learning re-sampling techniques for imbalanced datasets in 18F-FDG PET-based radiomics model on prognostication performance in cohorts of head and neck cancer patients
verfasst von: Chenyi Xie
Richard Du
Joshua WK Ho
Herbert H Pang
Keith WH Chiu
Elaine YP Lee
Varut Vardhanabhuti
Publikationsdatum: 06.04.2020
Verlag: Springer Berlin Heidelberg
Erschienen in: European Journal of Nuclear Medicine and Molecular Imaging / Ausgabe 12/2020
Print ISSN: 1619-7070
Elektronische ISSN: 1619-7089
DOI: https://doi.org/10.1007/s00259-020-04756-4

Die Highlights vom Kongress des American College of Cardiology 2024

Springer Medizin

Effect of machine learning re-sampling techniques for imbalanced datasets in ¹⁸F-FDG PET-based radiomics model on prognostication performance in cohorts of head and neck cancer patients

Abstract

Purpose

Methods

Results

Conclusion

Die Highlights vom Kongress des American College of Cardiology 2024

Springer Medizin

Abstract

Purpose

Methods

Results

Conclusion

Bitte loggen Sie sich ein, um Zugang zu diesem Inhalt zu erhalten

Weitere Artikel der Ausgabe 12/2020

Incorporation of miPSMA score for interpretation of 68Ga PSMA PET/CT scans for standardization and reproducibility of studies

Whole liver segmentation based on deep learning and manual adjustment for clinical use in SIRT

Correction to: Regional [18F]flortaucipir PET is more closely associated with disease severity than CSF p-tau in Alzheimer’s disease

18F-FDG muscular superscan associated with lipid storage myopathy

Reliable quantification of 18F-GE-180 PET neuroinflammation studies using an individually scaled population-based input function or late tissue-to-blood ratio

Development and characterization of CD54-targeted immunoPET imaging in solid tumors