Abstract
Machine Learning application on clinical data in order to support diagnosis and prognostic evaluation arouses growing interest in scientific community. However, choice of right algorithm to use was fundamental to perform reliable and robust classification. Our study aimed to explore if different kinds of Machine Learning technique could be effective to support early diagnosis of Multiple Sclerosis and which of them presented best performance in distinguishing Multiple Sclerosis patients from control subjects. We selected following algorithms: Random Forest, Support Vector Machine, Naïve-Bayes, K-nearest-neighbor and Artificial Neural Network. We applied the Independent Component Analysis to resting-state functional-MRI sequence to identify brain networks. We found 15 networks, from which we extracted the mean signals used into classification. We performed feature selection tasks in all algorithms to obtain the most important variables. We showed that best discriminant network between controls and early Multiple Sclerosis, was the sensori-motor I, according to early manifestation of motor/sensorial deficits in Multiple Sclerosis. Moreover, in classification performance, Random Forest and Support Vector Machine showed same 5-fold cross-validation accuracies (85.7%) using only this network, resulting to be best approaches. We believe that these findings could represent encouraging step toward the translation to clinical diagnosis and prognosis.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
ICASSO Toolbox, http://research.ics.aalto.fi/ica/icasso/
References
Bendfeldt, K., Klöppel, S., Nichols, T. E., Smieskova, R., Kuster, P., Traud, S., et al. (2012). Multivariate pattern classification of gray matter pathology in multiple sclerosis. Neuroimage, 60(1), 400–408.
Bhuvaneswari, P., & Kumar, J. S. (2013). Support vector machine technique for EEG signals. International Journal of Computer Applications, 63(13).
Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.
Brett, M., Anton, J. L., Valabregue, R., & Poline, J. B. (2002). Region of interest analysis using an SPM toolbox. In 8th international conference on functional mapping of the human brain. Japan: Sendai.
Briggs, F. B., Bartlett, S. E., Goldstein, B. A., Wang, J., McCauley, J. L., Zuvich, R. L., et al. (2010). Evidence for CRHR1 in multiple sclerosis using supervised machine learning and meta-analysis in 12 566 individuals. Human Molecular Genetics, 19(21), 4286–4295.
Calhoun, V. D., Liu, J., & Adali, T. (2009). A review of group ICA for fMRI data and ICA for joint inference of imaging, genetic, and ERP data. Neuroimage, 45, S163–S172.
Chan, J. C. W., Chan, K. P., & Yeh, A. G. O. (2001). Detecting the nature of change in an urban environment: A comparison of machine learning algorithms. Photogrammetric Engineering and Remote Sensing, 67(2), 213–226.
Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297.
Dayhoff, J. E., & DeLeo, J. M. (2001). Artificial neural networks. Cancer, 91(S8), 1615–1635.
Douglas, P. K., Harris, S., Yuille, A., & Cohen, M. S. (2011). Performance comparison of machine learning algorithms and number of independent components used in fMRI decoding of belief vs. disbelief. Neuroimage, 56(2), 544–553.
Formisano, E., De Martino, F., Valente, G. (2008). Multivariate analysis of fMRI time series: Classification and regression of brain responses using machine learning. Magnetic Resonance Imaging, 26(7), 921–934.
Gevrey, M., Dimopoulos, I., & Lek, S. (2006). Two-way interaction of input variables in the sensitivity analysis of neural network models. Ecological Modelling, 195(1), 43–50.
Goldstein, B. A., Hubbard, A. E., Cutler, A., & Barcellos, L. F. (2010). An application of random forests to a genome-wide association dataset: Methodological considerations & new findings. BMC Genetics, 11(1), 49.
Guyon, I., & Elisseeff, A. (2003). An introduction to variable and feature selection. Journal of Machine Learning Research, 3(Mar), 1157–1182.
Guyon, I., Weston, J., Barnhill, S., & Vapnik, V. (2002). Gene selection for cancer classification using support vector machines. Machine Learning, 46(1–3), 389–422.
Himberg, J., Hyvärinen, A., & Esposito, F. (2004). Validating the independent components of neuroimaging time-series via clustering and visualization. NeuroImage, 22(3), 1214–1222.
Ion-Mărgineanu, A., Kocevar, G., Stamile, C., Sima, D. M., Durand-Dubief, F., Van Huffel, S., Sappey-Marinier, D. (2017). Machine learning approach for classifying multiple sclerosis courses by combining clinical data with lesion loads and magnetic resonance metabolic features. Frontiers in Neuroscience, 11, 398.
Keller, A., Leidinger, P., Lange, J., Borries, A., Schroers, H., Scheffler, et al. (2009). Multiple sclerosis: microRNA expression profiles accurately differentiate patients with relapsing-remitting disease from healthy controls. PLoS One, 4(10), e7440.
Kumar, M., & Thenmozhi, M. (2006). Forecasting stock index movement: A comparison of support vector machines and random forest. Indian Institute of Capital Markets 9th Capital Markets Conference. http://ssrn.com/abstract, =876544.
LaConte, S., Strother, S., Cherkassky, V., Anderson, J., & Hu, X. (2005). Support vector machines for temporal classification of block design fMRI data. NeuroImage, 26(2), 317–329.
Lemm, S., Blankertz, B., Dickhaus, T., & Müller, K. R. (2011). Introduction to machine learning for brain imaging. Neuroimage, 56(2), 387–399.
Liu, M., Wang, M., Wang, J., & Li, D. (2013). Comparison of random forest, support vector machine and back propagation neural network for electronic tongue data classification: Application to the recognition of orange beverage and Chinese vinegar. Sensors and Actuators B: Chemical, 177, 970–980.
Lowe, M. J., Beall, E. B., Sakaie, K. E., Koenig, K. A., Stone, L., Marrie, R. A., & Phillips, M. D. (2008). Resting state sensori-motor functional connectivity in multiple sclerosis inversely correlates with transcallosal motor pathway transverse diffusivity. Human Brain Mapping, 29(7), 818–827.
Mesaros, S., Rocca, M. A., Kacar, K., Kostic, J., Copetti, M., Stosic-Opincal, T., et al. (2012). Diffusion tensor MRI tractography and cognitive impairment in multiple sclerosis. Neurology, 78(13), 969–975.
Mezzapesa, D. M., Rocca, M. A., Rodegher, M., Comi, G., et al. (2008). Functional cortical changes of the sensorimotor network are associated with clinical recovery in multiple sclerosis. Human Brain Mapping, 29(5), 562–573.
Misaki, M., Kim, Y., Bandettini, P. A., & Kriegeskorte, N. (2010). Comparison of multivariate classifiers and response normalizations for pattern-information fMRI. Neuroimage, 53(1), 103–118.
Mourão-Miranda, J., Bokde, A. L., Born, C., Hampel, H., & Stetter, M. (2005). Classifying brain states and determining the discriminating activation patterns: Support vector machine on functional MRI data. NeuroImage, 28(4), 980–995.
Murphy, K. P. (2006). Naive bayes classifiers. University of British Columbia.
Pang, B., Lee, L., Vaithyanathan, S. (2002). Thumbs up?: sentiment classification using machine learning techniques. In Proceedings of the ACL-02 conference on Empirical methods in natural language processing-Volume 10 (pp. 79–86). Association for Computational Linguistics, https://doi.org/10.3115/1118693.1118704
Pereira, F., Mitchell, T., & Botvinick, M. (2009). Machine learning classifiers and fMRI: A tutorial overview. Neuroimage, 45(1), S199–S209.
Peterson, L. E. (2009). K-nearest neighbor. Scholarpedia, 4(2), 1883.
Poldrack, R. A. (2007). Region of interest analysis for fMRI. Social Cognitive and Affective Neuroscience, 2(1), 67–70.
Polman, C. H., Reingold, S. C., Banwell, B., Clanet, M., Cohen, J. A., Filippi, M., et al. (2011). Diagnostic criteria for multiple sclerosis: 2010 revisions to the McDonald criteria. Annals of Neurology, 69(2), 292–302.
Rachakonda, S., Egolf, E., Correa, N., Calhoun, V. (2007). Group ICA of fMRI toolbox (GIFT) manual. Dostupné z http://www.nitrc.org/docman/view. Php/55/295/v1. 3d_ GIFTManual pdf [cit. 2011-11-5].
Refaeilzadeh, P., Tang, L., & Liu, H. (2009). Cross-validation (pp. 532–538). Springer US: In Encyclopedia of database systems.
Richiardi, J., Gschwind, M., Simioni, S., Annoni, J. M., Greco, B., Hagmann, P., et al. (2012). Classifying minimally disabled multiple sclerosis patients from resting state functional connectivity. Neuroimage, 62(3), 2021–2033.
Rocca, M. A., Absinta, M., Valsasina, P., Ciccarelli, O., Marino, S., Rovira, A., et al. (2009). Abnormal connectivity of the sensori-motor network in patients with MS: A multicenter fMRI study. Human Brain Mapping, 30(8), 2412–2425.
Sbardella, E., Petsas, N., Tona, F., and Pantano, P. (2015). Resting-State fMRI in MS: General Concepts and Brief Overview of Its Application. Biomed Res Int, 212693.
Sivapriya, T. R., Kamal, A. R., Thangaiah, P. (2015). Ensemble Merit Merge Feature Selection for Enhanced Multinomial Classification in Alzheimer’s Dementia. Computational and mathematical methods in medicine, 2015.
Statnikov, A., Wang, L., & Aliferis, C. F. (2008). A comprehensive comparison of random forests and support vector machines for microarray-based cancer classification. BMC Bioinformatics, 9(1), 319.
Team, R. C. (2000). R language definition. Vienna, Austria: R foundation for statistical computing. http://www.R-project.org/
The CAMMS223 Trial Investigators. (2008). Alemtuzumab vs. interferon Beta-1a in early multiple sclerosis. The New England Journal of Medicine, 359, 1786–1801.
Ulrich, R., Kalkuhl, A., Deschl, U., & Baumgärtner, W. (2010). Machine learning approach identifies new pathways associated with demyelination in a viral model of multiple sclerosis. Journal of Cellular and Molecular Medicine, 14(1–2), 434–448.
Wang, Z., Childress, A. R., Wang, J., & Detre, J. A. (2007). Support vector machine learning-based fMRI data group analysis. NeuroImage, 36(4), 1139–1151.
Yamamoto, D., Arimura, H., Kakeda, S., Magome, T., Yamashita, Y., Toyofuku, F., et al. (2010). Computer-aided detection of multiple sclerosis lesions in brain magnetic resonance images: False positive reduction scheme consisted of rule-based, level set method, and support vector machine. Computerized Medical Imaging and Graphics, 34(5), 404–413.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Disclosure
This study was not financially supported.
Conflict of interest
All of the authors reported no biomedical financial interests or potential conflicts of interest.
Ethical approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Rights and permissions
About this article
Cite this article
Saccà, V., Sarica, A., Novellino, F. et al. Evaluation of machine learning algorithms performance for the prediction of early multiple sclerosis from resting-state FMRI connectivity data. Brain Imaging and Behavior 13, 1103–1114 (2019). https://doi.org/10.1007/s11682-018-9926-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11682-018-9926-9