Skip to main content
Erschienen in: International Journal of Computer Assisted Radiology and Surgery 2/2024

10.10.2023 | Original Article

A preliminary exploration into top-down and bottom-up deep-learning approaches to localising neuro-interventional point targets in volumetric MRI

verfasst von: Enora Giffard, Pierre Jannin, John S. H. Baxter

Erschienen in: International Journal of Computer Assisted Radiology and Surgery | Ausgabe 2/2024

Einloggen, um Zugang zu erhalten

Abstract

Purpose

Point localisation is a critical aspect of many interventional planning procedures, specifically representing anatomical regions of interest or landmarks as individual points. This could be seen as analogous to the problem of visual search in cognitive psychology, in which this search is performed either: bottom-up, constructing increasingly abstract and coarse-resolution features over the entire image; or top-down, using contextual cues from the entire image to refine the scope of the region being investigated. Traditional convolutional neural networks use the former, but it is not clear if this is optimal. This article is a preliminary investigation as to how this motivation affects 3D point localisation in neuro-interventional planning.

Methods

Two neuro-imaging datasets were collected: one for cortical point localisation for repetitive transcranial magnetic stimulation and the other for sub-cortical anatomy localisation for deep brain stimulation. Four different frameworks were developed using top-down versus bottom-up paradigms as well as representing points as co-ordinates or heatmaps. These networks were applied to point localisation for transcranial magnetic stimulation and subcortical anatomy localisation. These networks were evaluated using cross-validation and a varying number of training datasets to analyse their sensitivity to quantity of training data.

Results

Each network shows increasing performance as the amount of available training data increases, with the co-ordinate-based top-down network consistently outperforming the others. Specifically, the top-down architectures tend to outperform the bottom-up ones. An analysis of their memory consumption also encourages the top-down co-ordinate based architecture as it requires significantly less memory than either bottom-up architectures or those representing their predictions via heatmaps.

Conclusion

This paper is a preliminary foray into a fundamental aspect of machine learning architectural design: that of the top-down/bottom-up divide from cognitive psychology. Although there are additional considerations within the particular architectures investigated that could affect these results and the number of architectures investigated is limited, our results do indicate that the less commonly used top-down paradigm could lead to more efficient and effective architectures in the future.
Fußnoten
1
http://​www.​itksnap.​org/​pmwiki/​pmwiki.​php?​n=​Convert3D.​Convert3D, The specific commands used were swapdim, -resample-mm, and -pad-to.
 
Literatur
1.
Zurück zum Zitat He K, Sun J (2015) Convolutional neural networks at constrained time cost. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5353–5360 He K, Sun J (2015) Convolutional neural networks at constrained time cost. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5353–5360
2.
Zurück zum Zitat Qiao S, Lin Z, Zhang J, Yuille A L (2019) Neural rejuvenation: improving deep network training by enhancing computational resource utilization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 61–71 Qiao S, Lin Z, Zhang J, Yuille A L (2019) Neural rejuvenation: improving deep network training by enhancing computational resource utilization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 61–71
3.
Zurück zum Zitat Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 580–587 Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 580–587
4.
Zurück zum Zitat Girshick R (2015) Fast r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp. 1440–1448 Girshick R (2015) Fast r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp. 1440–1448
5.
Zurück zum Zitat Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, vol. 28, pp. 91–99 Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, vol. 28, pp. 91–99
6.
Zurück zum Zitat Sugimori H, Kawakami M (2019) Automatic detection of a standard line for brain magnetic resonance imaging using deep learning. In: Applied Sciences, vol. 9, 3849 Sugimori H, Kawakami M (2019) Automatic detection of a standard line for brain magnetic resonance imaging using deep learning. In: Applied Sciences, vol. 9, 3849
7.
Zurück zum Zitat Yang X, Tang WT, Tjio G, Yeo SY, Su Y (2020) Automatic detection of anatomical landmarks in brain MR scanning using multi-task deep neural networks. In: Neurocomputing, vol. 396, pp. 514–521 Yang X, Tang WT, Tjio G, Yeo SY, Su Y (2020) Automatic detection of anatomical landmarks in brain MR scanning using multi-task deep neural networks. In: Neurocomputing, vol. 396, pp. 514–521
8.
Zurück zum Zitat Gohel B, Kumar L, Shah D (2023) Deep learning-based automated localisation of anterior commissure and posterior commissure landmarks in 3d space from three-plane 2d mri localiser slices of the brain. In: Procedia Computer Science, vol. 218, pp. 1027–1032 Gohel B, Kumar L, Shah D (2023) Deep learning-based automated localisation of anterior commissure and posterior commissure landmarks in 3d space from three-plane 2d mri localiser slices of the brain. In: Procedia Computer Science, vol. 218, pp. 1027–1032
9.
Zurück zum Zitat Baxter JS, Bui QA, Maguet E, Croci S, Delmas A, Lefaucheur J-P, Bredoux L, Jannin P (2021) Automatic cortical target point localisation in MRI for transcranial magnetic stimulation via a multi-resolution convolutional neural network. In: International Journal of Computer Assisted Radiology and Surgery, vol. 16, pp.1077–1087 Baxter JS, Bui QA, Maguet E, Croci S, Delmas A, Lefaucheur J-P, Bredoux L, Jannin P (2021) Automatic cortical target point localisation in MRI for transcranial magnetic stimulation via a multi-resolution convolutional neural network. In: International Journal of Computer Assisted Radiology and Surgery, vol. 16, pp.1077–1087
10.
Zurück zum Zitat Foulsham T, Chapman C, Nasiopoulos E, Kingstone A (2014) Top-down and bottom-up aspects of active search in a real-world environment. In: Canadian Journal of Experimental Psychology, vol. 68, pp. 8–19 Foulsham T, Chapman C, Nasiopoulos E, Kingstone A (2014) Top-down and bottom-up aspects of active search in a real-world environment. In: Canadian Journal of Experimental Psychology, vol. 68, pp. 8–19
11.
Zurück zum Zitat Li S, Gong Q, Li H, Chen S, Liu Y, Ruan G, Zhu L, Liu L, Chen H (2022) Automatic location scheme of anatomical landmarks in 3d head MRI based on the scale attention hourglass network. Comput Methods Progr Biomed 214:106564CrossRef Li S, Gong Q, Li H, Chen S, Liu Y, Ruan G, Zhu L, Liu L, Chen H (2022) Automatic location scheme of anatomical landmarks in 3d head MRI based on the scale attention hourglass network. Comput Methods Progr Biomed 214:106564CrossRef
12.
Zurück zum Zitat Lester H, Arridge SR (1999) A survey of hierarchical non-linear medical image registration. Pattern Recogn 32(1):129–149ADSCrossRef Lester H, Arridge SR (1999) A survey of hierarchical non-linear medical image registration. Pattern Recogn 32(1):129–149ADSCrossRef
13.
14.
Zurück zum Zitat Balconi M (2013) Dorsolateral prefrontal cortex, working memory and episodic memory processes: insight through transcranial magnetic stimulation techniques. Neurosci Bull 29:381–389CrossRefPubMedPubMedCentral Balconi M (2013) Dorsolateral prefrontal cortex, working memory and episodic memory processes: insight through transcranial magnetic stimulation techniques. Neurosci Bull 29:381–389CrossRefPubMedPubMedCentral
15.
Zurück zum Zitat Hamid P, Malik BH, Hussain ML (2019) Noninvasive transcranial magnetic stimulation (TMS) in chronic refractory pain: a systematic review. Cureus 11(10):e6019PubMedPubMedCentral Hamid P, Malik BH, Hussain ML (2019) Noninvasive transcranial magnetic stimulation (TMS) in chronic refractory pain: a systematic review. Cureus 11(10):e6019PubMedPubMedCentral
16.
Zurück zum Zitat Sparing R, Buelte D, Meister IG, Pauš T, Fink GR (2008) Transcranial magnetic stimulation and the challenge of coil placement: a comparison of conventional and stereotaxic neuronavigational strategies. Hum Brain Mapp 29(1):82–96CrossRefPubMed Sparing R, Buelte D, Meister IG, Pauš T, Fink GR (2008) Transcranial magnetic stimulation and the challenge of coil placement: a comparison of conventional and stereotaxic neuronavigational strategies. Hum Brain Mapp 29(1):82–96CrossRefPubMed
17.
Zurück zum Zitat Siebner HR, Hartwigsen G, Kassuba T, Rothwell JC (2009) How does transcranial magnetic stimulation modify neuronal activity in the brain? Implications for studies of cognition. Cortex 45(9):1035–1042CrossRefPubMedPubMedCentral Siebner HR, Hartwigsen G, Kassuba T, Rothwell JC (2009) How does transcranial magnetic stimulation modify neuronal activity in the brain? Implications for studies of cognition. Cortex 45(9):1035–1042CrossRefPubMedPubMedCentral
18.
Zurück zum Zitat Middlebrooks E, Domingo R, Vivas-Buitrago T, Okromelidze L, Tsuboi T, Wong J, Eisinger R, Almeida L, Burns M, Horn A et al (2020) Neuroimaging advances in deep brain stimulation: review of indications, anatomy, and brain connectomics. Am J Neuroradiol 41(9):1558–1568 Middlebrooks E, Domingo R, Vivas-Buitrago T, Okromelidze L, Tsuboi T, Wong J, Eisinger R, Almeida L, Burns M, Horn A et al (2020) Neuroimaging advances in deep brain stimulation: review of indications, anatomy, and brain connectomics. Am J Neuroradiol 41(9):1558–1568
19.
Zurück zum Zitat Baxter JS, Jannin P (2023) Validation in the age of machine learning: a framework for describing validation with examples in transcranial magnetic stimulation and deep brain stimulation. Intell. -Based Med. 7:100090 Baxter JS, Jannin P (2023) Validation in the age of machine learning: a framework for describing validation with examples in transcranial magnetic stimulation and deep brain stimulation. Intell. -Based Med. 7:100090
20.
Zurück zum Zitat Haegelen C, Coupé P, Fonov V, Guizard N, Jannin P, Morandi X, Collins DL (2013) Automated segmentation of basal ganglia and deep brain structures in MRI of Parkinson’s disease. Int J Comput Assist Radiol Surg 8(1):99–110CrossRefPubMed Haegelen C, Coupé P, Fonov V, Guizard N, Jannin P, Morandi X, Collins DL (2013) Automated segmentation of basal ganglia and deep brain structures in MRI of Parkinson’s disease. Int J Comput Assist Radiol Surg 8(1):99–110CrossRefPubMed
21.
Zurück zum Zitat Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3431–3440 Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3431–3440
22.
Zurück zum Zitat Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention-MICCAI, 18th International Conference, Munich, Proceedings, Part III 18. Springer 2015, pp. 234–241 Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention-MICCAI, 18th International Conference, Munich, Proceedings, Part III 18. Springer 2015, pp. 234–241
23.
Zurück zum Zitat Ranftl R, Bochkovskiy A, Koltun V (2021) Vision transformers for dense prediction. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 12 179–12 188 Ranftl R, Bochkovskiy A, Koltun V (2021) Vision transformers for dense prediction. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 12 179–12 188
24.
Zurück zum Zitat Hatamizadeh A, Nath V, Tang Y, Yang D, Roth HR, Xu D (2021) Swin unetr: swin transformers for semantic segmentation of brain tumors in MRI images. In: Brainlesion: glioma, multiple sclerosis, stroke and traumatic brain injuries: 7th international workshop, BrainLes. Held in Conjunction with MICCAI 2021, Virtual Event, Revised Selected Papers. Part I. Springer 2022:272–284 Hatamizadeh A, Nath V, Tang Y, Yang D, Roth HR, Xu D (2021) Swin unetr: swin transformers for semantic segmentation of brain tumors in MRI images. In: Brainlesion: glioma, multiple sclerosis, stroke and traumatic brain injuries: 7th international workshop, BrainLes. Held in Conjunction with MICCAI 2021, Virtual Event, Revised Selected Papers. Part I. Springer 2022:272–284
25.
Zurück zum Zitat Baxter JS, Jannin P (2022) Combining simple interactivity and machine learning: a separable deep learning approach to subthalamic nucleus localization and segmentation in MRI for deep brain stimulation surgical planning. J Med Imaging 9(4):045001CrossRef Baxter JS, Jannin P (2022) Combining simple interactivity and machine learning: a separable deep learning approach to subthalamic nucleus localization and segmentation in MRI for deep brain stimulation surgical planning. J Med Imaging 9(4):045001CrossRef
Metadaten
Titel
A preliminary exploration into top-down and bottom-up deep-learning approaches to localising neuro-interventional point targets in volumetric MRI
verfasst von
Enora Giffard
Pierre Jannin
John S. H. Baxter
Publikationsdatum
10.10.2023
Verlag
Springer International Publishing
Erschienen in
International Journal of Computer Assisted Radiology and Surgery / Ausgabe 2/2024
Print ISSN: 1861-6410
Elektronische ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-023-03023-9

Weitere Artikel der Ausgabe 2/2024

International Journal of Computer Assisted Radiology and Surgery 2/2024 Zur Ausgabe

Darf man die Behandlung eines Neonazis ablehnen?

08.05.2024 Gesellschaft Nachrichten

In einer Leseranfrage in der Zeitschrift Journal of the American Academy of Dermatology möchte ein anonymer Dermatologe bzw. eine anonyme Dermatologin wissen, ob er oder sie einen Patienten behandeln muss, der eine rassistische Tätowierung trägt.

Ein Drittel der jungen Ärztinnen und Ärzte erwägt abzuwandern

07.05.2024 Klinik aktuell Nachrichten

Extreme Arbeitsverdichtung und kaum Supervision: Dr. Andrea Martini, Sprecherin des Bündnisses Junge Ärztinnen und Ärzte (BJÄ) über den Frust des ärztlichen Nachwuchses und die Vorteile des Rucksack-Modells.

Endlich: Zi zeigt, mit welchen PVS Praxen zufrieden sind

IT für Ärzte Nachrichten

Darauf haben viele Praxen gewartet: Das Zi hat eine Liste von Praxisverwaltungssystemen veröffentlicht, die von Nutzern positiv bewertet werden. Eine gute Grundlage für wechselwillige Ärztinnen und Psychotherapeuten.

Akuter Schwindel: Wann lohnt sich eine MRT?

28.04.2024 Schwindel Nachrichten

Akuter Schwindel stellt oft eine diagnostische Herausforderung dar. Wie nützlich dabei eine MRT ist, hat eine Studie aus Finnland untersucht. Immerhin einer von sechs Patienten wurde mit akutem ischämischem Schlaganfall diagnostiziert.

Update Radiologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.