Skip to main content
Erschienen in: Journal of Digital Imaging 1/2019

23.07.2018

AdaptAhead Optimization Algorithm for Learning Deep CNN Applied to MRI Segmentation

verfasst von: Farnaz Hoseini, Asadollah Shahbahrami, Peyman Bayat

Erschienen in: Journal of Imaging Informatics in Medicine | Ausgabe 1/2019

Einloggen, um Zugang zu erhalten

Abstract

Deep learning is one of the subsets of machine learning that is widely used in artificial intelligence (AI) field such as natural language processing and machine vision. The deep convolution neural network (DCNN) extracts high-level concepts from low-level features and it is appropriate for large volumes of data. In fact, in deep learning, the high-level concepts are defined by low-level features. Previously, in optimization algorithms, the accuracy achieved for network training was less and high-cost function. In this regard, in this study, AdaptAhead optimization algorithm was developed for learning DCNN with robust architecture in relation to the high volume data. The proposed optimization algorithm was validated in multi-modality MR images of BRATS 2015 and BRATS 2016 data sets. Comparison of the proposed optimization algorithm with other commonly used methods represents the improvement of the performance of the proposed optimization algorithm on the relatively large dataset. Using the Dice similarity metric, we report accuracy results on the BRATS 2015 and BRATS 2016 brain tumor segmentation challenge dataset. Results showed that our proposed algorithm is significantly more accurate than other methods as a result of its deep and hierarchical extraction.
Literatur
1.
Zurück zum Zitat LeCun Y, Bengio Y, Hinton G: Deep learning. Nature 521(7553):436–444, 2015CrossRef LeCun Y, Bengio Y, Hinton G: Deep learning. Nature 521(7553):436–444, 2015CrossRef
2.
Zurück zum Zitat Schmidhuber J: Deep learning in neural networks: An overview. Neural networks 61:85–117, 2015CrossRefPubMed Schmidhuber J: Deep learning in neural networks: An overview. Neural networks 61:85–117, 2015CrossRefPubMed
3.
Zurück zum Zitat Krizhevsky A, Sutskever I, Hinton GE: Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems:1097–1105, 2012 Krizhevsky A, Sutskever I, Hinton GE: Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems:1097–1105, 2012
4.
Zurück zum Zitat Karpathy A, Toderici G, Shetty S, Leung T, Sukthankar R, Fei-Fei L: Large-scale video classification with convolutional neural networks. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2014, (pp. 1725–1732). Karpathy A, Toderici G, Shetty S, Leung T, Sukthankar R, Fei-Fei L: Large-scale video classification with convolutional neural networks. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2014, (pp. 1725–1732).
5.
Zurück zum Zitat Sainath TN, Mohamed AR, Kingsbury B, Ramabhadran B: Deep convolutional neural networks for LVCSR. In Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on IEEE. 2013, pp. 8614–8618 Sainath TN, Mohamed AR, Kingsbury B, Ramabhadran B: Deep convolutional neural networks for LVCSR. In Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on IEEE. 2013, pp. 8614–8618
6.
Zurück zum Zitat Vincent P, Larochelle H, Bengio Y, Manzagol PA: Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning. ACM. 2008pp. 1096–1103 Vincent P, Larochelle H, Bengio Y, Manzagol PA: Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning. ACM. 2008pp. 1096–1103
7.
Zurück zum Zitat Schaul T, Antonoglou I, Silver D: Unit tests for stochastic optimization. arXiv preprint arXiv:1312.6055, 2013 Schaul T, Antonoglou I, Silver D: Unit tests for stochastic optimization. arXiv preprint arXiv:1312.6055, 2013
8.
Zurück zum Zitat Zhang T: Solving large scale linear prediction problems using stochastic gradient descent algorithms. In Proceedings of the twenty-first international conference on Machine learning. ACM. 2004, p. 116 Zhang T: Solving large scale linear prediction problems using stochastic gradient descent algorithms. In Proceedings of the twenty-first international conference on Machine learning. ACM. 2004, p. 116
9.
Zurück zum Zitat Sutskever I, Martens J, Dahl G, Hinton G: On the importance of initialization and momentum in deep learning. In International conference on machine learning, 2013, (pp. 1139–114) Sutskever I, Martens J, Dahl G, Hinton G: On the importance of initialization and momentum in deep learning. In International conference on machine learning, 2013, (pp. 1139–114)
10.
Zurück zum Zitat Nesterov Y: A method of solving a convex programming problem with convergence rate O (1/k2). In Soviet Mathematics Doklady (Vol. 27, No. 2, pp. 372–376), 1983 Nesterov Y: A method of solving a convex programming problem with convergence rate O (1/k2). In Soviet Mathematics Doklady (Vol. 27, No. 2, pp. 372–376), 1983
11.
Zurück zum Zitat Polyak BT: Some methods of speeding up the convergence of iteration methods. USSR Comput Math Math Phys 4(5):1–17, 1964CrossRef Polyak BT: Some methods of speeding up the convergence of iteration methods. USSR Comput Math Math Phys 4(5):1–17, 1964CrossRef
12.
Zurück zum Zitat Duchi J, Hazan E, Singer Y: Adaptive subgradient methods for online learning and stochastic optimization. J Mach Learn Res 12:2121–2159, 2011 Duchi J, Hazan E, Singer Y: Adaptive subgradient methods for online learning and stochastic optimization. J Mach Learn Res 12:2121–2159, 2011
13.
Zurück zum Zitat Kingma D, Ba J: Adam: A method for stochastic optimization. In 3rd International Conference for Learning Representations, 2014, (pp. 1–13) Kingma D, Ba J: Adam: A method for stochastic optimization. In 3rd International Conference for Learning Representations, 2014, (pp. 1–13)
14.
Zurück zum Zitat Hoseini F, Shahbahrami, Bayat P: A hybrid optimization algorithm for learning deep models. Journal of Advances in Computer Research 9(4):1–13, 2018 Hoseini F, Shahbahrami, Bayat P: A hybrid optimization algorithm for learning deep models. Journal of Advances in Computer Research 9(4):1–13, 2018
15.
Zurück zum Zitat Bottou L, Bousquet O: The tradeoffs of large scale learning. In International Conference of Advances in neural information processing systems, 2008, (pp. 161–168) Bottou L, Bousquet O: The tradeoffs of large scale learning. In International Conference of Advances in neural information processing systems, 2008, (pp. 161–168)
16.
Zurück zum Zitat Saad D,Solla, SA: Exact solution for on-line learning in multilayer neural networks. Phys Rev Lett 74(21):4337, 1995 Saad D,Solla, SA: Exact solution for on-line learning in multilayer neural networks. Phys Rev Lett 74(21):4337, 1995
17.
Zurück zum Zitat Nesterov Y: Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM J Optim 22(2):341–362, 2012 Nesterov Y: Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM J Optim 22(2):341–362, 2012
18.
Zurück zum Zitat Huang FJ, Boureau YL, LeCun Y: Unsupervised learning of invariant feature hierarchies with applications to object recognition. In International Conference of Computer Vision and Pattern Recognition, 2007, (pp. 1–8) Huang FJ, Boureau YL, LeCun Y: Unsupervised learning of invariant feature hierarchies with applications to object recognition. In International Conference of Computer Vision and Pattern Recognition, 2007, (pp. 1–8)
19.
Zurück zum Zitat Dozat T: Incorporating Nesterov momentum into Adam, 2016 Dozat T: Incorporating Nesterov momentum into Adam, 2016
20.
Zurück zum Zitat Zikic D, Ioannou Y, Brown M, Criminisi A: Segmentation of brain tumor tissues with convolutional neural networks. In International Conference of Proceedings MICCAI-BRATS, 2014, (pp. 36–39) Zikic D, Ioannou Y, Brown M, Criminisi A: Segmentation of brain tumor tissues with convolutional neural networks. In International Conference of Proceedings MICCAI-BRATS, 2014, (pp. 36–39)
21.
Zurück zum Zitat Long J, Shelhamer E, Darrell T: Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3431–3440), 2015 Long J, Shelhamer E, Darrell T: Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3431–3440), 2015
22.
Zurück zum Zitat Dvořák P, Menze B: Local Structure Prediction with Convolutional Neural Networks for Multimodal Brain Tumor Segmentation. In International MICCAI Workshop on Medical Computer Vision (pp. 59–71). Springer International Publishin, 2015 Dvořák P, Menze B: Local Structure Prediction with Convolutional Neural Networks for Multimodal Brain Tumor Segmentation. In International MICCAI Workshop on Medical Computer Vision (pp. 59–71). Springer International Publishin, 2015
23.
Zurück zum Zitat Pereira S, Pinto A, Alves V, Silva CA: Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans Med Imaging 35(5):1240–1251, 2016CrossRefPubMed Pereira S, Pinto A, Alves V, Silva CA: Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans Med Imaging 35(5):1240–1251, 2016CrossRefPubMed
24.
Zurück zum Zitat Havaei M, Davy A, Warde-Farley D, Biard A, Courville A, Bengio Y, Pal C, Jodoin PM, Larochelle H: Brain tumor segmentation with deep neural networks. Medical image analysis 35:18–31, 2017CrossRefPubMed Havaei M, Davy A, Warde-Farley D, Biard A, Courville A, Bengio Y, Pal C, Jodoin PM, Larochelle H: Brain tumor segmentation with deep neural networks. Medical image analysis 35:18–31, 2017CrossRefPubMed
25.
Zurück zum Zitat Havaei M., Guizard N., Larochelle H., Jodoin PM. (2016) Deep learning trends for focal brain pathology segmentation in MRI. In: Holzinger A Ed. Machine Learning for Health Informatics. Lecture Notes in Computer Science (Vol. 9605). Cham, Springer International Publishing, 2016 Havaei M., Guizard N., Larochelle H., Jodoin PM. (2016) Deep learning trends for focal brain pathology segmentation in MRI. In: Holzinger A Ed. Machine Learning for Health Informatics. Lecture Notes in Computer Science (Vol. 9605). Cham, Springer International Publishing, 2016
Metadaten
Titel
AdaptAhead Optimization Algorithm for Learning Deep CNN Applied to MRI Segmentation
verfasst von
Farnaz Hoseini
Asadollah Shahbahrami
Peyman Bayat
Publikationsdatum
23.07.2018
Verlag
Springer International Publishing
Erschienen in
Journal of Imaging Informatics in Medicine / Ausgabe 1/2019
Print ISSN: 2948-2925
Elektronische ISSN: 2948-2933
DOI
https://doi.org/10.1007/s10278-018-0107-6

Weitere Artikel der Ausgabe 1/2019

Journal of Digital Imaging 1/2019 Zur Ausgabe

Update Radiologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.