Skip to main content
Erschienen in: International Journal of Computer Assisted Radiology and Surgery 5/2021

11.04.2021 | Original Article

Time-aware deep neural networks for needle tip localization in 2D ultrasound

verfasst von: Cosmas Mwikirize, Alvin B. Kimbowa, Sylvia Imanirakiza, Andrew Katumba, John L. Nosher, Ilker Hacihaliloglu

Erschienen in: International Journal of Computer Assisted Radiology and Surgery | Ausgabe 5/2021

Einloggen, um Zugang zu erhalten

Abstract

Purpose

Accurate placement of the needle is critical in interventions like biopsies and regional anesthesia, during which incorrect needle insertion can lead to procedure failure and complications. Therefore, ultrasound guidance is widely used to improve needle placement accuracy. However, at steep and deep insertions, the visibility of the needle is lost. Computational methods for automatic needle tip localization could improve the clinical success rate in these scenarios.

Methods

We propose a novel algorithm for needle tip localization during challenging ultrasound-guided insertions when the shaft may be invisible, and the tip has a low intensity. There are two key steps in our approach. First, we enhance the needle tip features in consecutive ultrasound frames using a detection scheme which recognizes subtle intensity variations caused by needle tip movement. We then employ a hybrid deep neural network comprising a convolutional neural network and long short-term memory recurrent units. The input to the network is a consecutive plurality of fused enhanced frames and the corresponding original B-mode frames, and this spatiotemporal information is used to predict the needle tip location.

Results

We evaluate our approach on an ex vivo dataset collected with in-plane and out-of-plane insertion of 17G and 22G needles in bovine, porcine, and chicken tissue, acquired using two different ultrasound systems. We train the model with 5000 frames from 42 video sequences. Evaluation on 600 frames from 30 sequences yields a tip localization error of \(0.52\pm 0.06\) mm and an overall inference time of 0.064 s (15 fps). Comparison against prior art on challenging datasets reveals a 30% improvement in tip localization accuracy.

Conclusion

The proposed method automatically models temporal dynamics associated with needle tip motion and is more accurate than state-of-the-art methods. Therefore, it has the potential for improving needle tip localization in challenging ultrasound-guided interventions.
Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Zurück zum Zitat Marhofer P, Fritsch G (2017) Safe performance of peripheral regional anaesthesia: the significance of ultrasound guidance. Anaesthesia 72(4):431–434CrossRef Marhofer P, Fritsch G (2017) Safe performance of peripheral regional anaesthesia: the significance of ultrasound guidance. Anaesthesia 72(4):431–434CrossRef
2.
Zurück zum Zitat Huang J, Li J, Wang H (2018) The principles and procedures of ultrasound-guided anesthesia techniques. Cureus 10(7):e2980PubMedPubMedCentral Huang J, Li J, Wang H (2018) The principles and procedures of ultrasound-guided anesthesia techniques. Cureus 10(7):e2980PubMedPubMedCentral
3.
Zurück zum Zitat Kim SY, Chung HW, Oh TS, Lee J (2017) Practical guidelines for ultrasound-guided core needle biopsy of soft-tissue lesions: transformation from beginner to specialist. Korean J Radiol 18(2):361–369CrossRef Kim SY, Chung HW, Oh TS, Lee J (2017) Practical guidelines for ultrasound-guided core needle biopsy of soft-tissue lesions: transformation from beginner to specialist. Korean J Radiol 18(2):361–369CrossRef
4.
Zurück zum Zitat Elsharkawy H, Babazade R, Kolli S, Kalagara H, Soliman ML (2016) The Infiniti plus ultrasound needle guidance system improves needle visualization during the placement of spinal anesthesia. Korean J Anesthesiol 69(4):417–419CrossRef Elsharkawy H, Babazade R, Kolli S, Kalagara H, Soliman ML (2016) The Infiniti plus ultrasound needle guidance system improves needle visualization during the placement of spinal anesthesia. Korean J Anesthesiol 69(4):417–419CrossRef
5.
Zurück zum Zitat Lu H, Li J, Lu Q, Bharat S, Erkamp R, Chen B, Drysdale J, Vignon F, Jain A (2014) A new sensor technology for 2D ultrasound-guided needle tracking. MICCAI 17(Pt. 2):389–396PubMed Lu H, Li J, Lu Q, Bharat S, Erkamp R, Chen B, Drysdale J, Vignon F, Jain A (2014) A new sensor technology for 2D ultrasound-guided needle tracking. MICCAI 17(Pt. 2):389–396PubMed
6.
Zurück zum Zitat Xia W, West S, Finlay M, Mari J, Ourselin S, David A, Desjardins A (2017) Looking beyond the imaging plane: 3D needle tracking with a linear array ultrasound probe. Sci Rep 7(1):3674CrossRef Xia W, West S, Finlay M, Mari J, Ourselin S, David A, Desjardins A (2017) Looking beyond the imaging plane: 3D needle tracking with a linear array ultrasound probe. Sci Rep 7(1):3674CrossRef
7.
Zurück zum Zitat Arif M, Moelker A, van Walsum T (2018) Needle tip visibility in 3D ultrasound images. Cardiovasc Intervent Radiol 41(1):145–152CrossRef Arif M, Moelker A, van Walsum T (2018) Needle tip visibility in 3D ultrasound images. Cardiovasc Intervent Radiol 41(1):145–152CrossRef
8.
Zurück zum Zitat Umbarje K, Tang R, Randhawa R, Sawka A, Vaghadia H (2013) Out-of-plane brachial plexus block with a novel SonixGPS (TM) needle tracking system. Anaesthesia 68:433–434CrossRef Umbarje K, Tang R, Randhawa R, Sawka A, Vaghadia H (2013) Out-of-plane brachial plexus block with a novel SonixGPS (TM) needle tracking system. Anaesthesia 68:433–434CrossRef
9.
Zurück zum Zitat Gadsden J, Latmore M, Levine DM (2015) Evaluation of the eZono 4000 with eZGuide for ultrasound-guided procedures. Expert Rev Med Dev 12:251–261CrossRef Gadsden J, Latmore M, Levine DM (2015) Evaluation of the eZono 4000 with eZGuide for ultrasound-guided procedures. Expert Rev Med Dev 12:251–261CrossRef
10.
Zurück zum Zitat McVicar J, Niazi AU, Murgatroyd H, Chin KJ, Chan VW (2015) Novice performance of ultrasound-guided needling skills: effect of a needle guidance system. Reg Anesth Pain Med 40:150–153CrossRef McVicar J, Niazi AU, Murgatroyd H, Chin KJ, Chan VW (2015) Novice performance of ultrasound-guided needling skills: effect of a needle guidance system. Reg Anesth Pain Med 40:150–153CrossRef
11.
Zurück zum Zitat Najafi M, Abolmaesumi P, Rohling R (2015) Single-camera closed-form real-time needle tracking for ultrasound-guided needle insertion. Ultrasound Med Biol 41:2663–2676CrossRef Najafi M, Abolmaesumi P, Rohling R (2015) Single-camera closed-form real-time needle tracking for ultrasound-guided needle insertion. Ultrasound Med Biol 41:2663–2676CrossRef
12.
Zurück zum Zitat Xia W, Mari JM, West SJ, Ginsberg Y, David AL, Ourselin S, Desjardins AE (2015) In-plane ultrasonic needle tracking using a fiber-optic hydrophone. Med Phys 42:5983–5991CrossRef Xia W, Mari JM, West SJ, Ginsberg Y, David AL, Ourselin S, Desjardins AE (2015) In-plane ultrasonic needle tracking using a fiber-optic hydrophone. Med Phys 42:5983–5991CrossRef
13.
Zurück zum Zitat Stolka PJ, Foroughi P, Rendina M, Weiss CR, Hager GD, Boctor EM (2014) Needle guidance using handheld stereo vision and projection for ultrasound-based interventions. MICCAI 17(Pt.2):684–691PubMed Stolka PJ, Foroughi P, Rendina M, Weiss CR, Hager GD, Boctor EM (2014) Needle guidance using handheld stereo vision and projection for ultrasound-based interventions. MICCAI 17(Pt.2):684–691PubMed
14.
Zurück zum Zitat Neubash Z, Shoham M (2010) Ultrasound-guided robot for flexible needle steering. IEEE TBME 57(4):799–805 Neubash Z, Shoham M (2010) Ultrasound-guided robot for flexible needle steering. IEEE TBME 57(4):799–805
15.
Zurück zum Zitat Welleweerd MK, Siepel FJ, Groenhius V, Veltman J, Stramigioli S (2020) Design of an end effector for robot-assisted ultrasound-guided breast biopsies. Int J Comput Assist Radiol Surg 15:681–690CrossRef Welleweerd MK, Siepel FJ, Groenhius V, Veltman J, Stramigioli S (2020) Design of an end effector for robot-assisted ultrasound-guided breast biopsies. Int J Comput Assist Radiol Surg 15:681–690CrossRef
16.
Zurück zum Zitat Hatt CR, Ng G, Parthasarathy V (2015) Enhanced needle localization in ultrasound using beam steering and learning-based segmentation. Comput Med Imaging Graph 41:46–54CrossRef Hatt CR, Ng G, Parthasarathy V (2015) Enhanced needle localization in ultrasound using beam steering and learning-based segmentation. Comput Med Imaging Graph 41:46–54CrossRef
17.
Zurück zum Zitat Beigi P, Rohling R, Salcudean S, Ng G (2017) CASPER: computer-aided segmentation of imperceptible motion—a learning-based tracking of an invisible needle in ultrasound. Int J Comput Assist Radiol Surg 12(11):1857–1866CrossRef Beigi P, Rohling R, Salcudean S, Ng G (2017) CASPER: computer-aided segmentation of imperceptible motion—a learning-based tracking of an invisible needle in ultrasound. Int J Comput Assist Radiol Surg 12(11):1857–1866CrossRef
18.
Zurück zum Zitat Pourtaherian A, Ghazvinian Zanjani F, Zinger S, Mihajlovic N, Ng G, Korsten H, With P (2018) Robust and semantic needle detection in 3D ultrasound using orthogonal-plane convolutional neural networks. Int J Comput Assist Radiol Surg 13(9):1321–1333CrossRef Pourtaherian A, Ghazvinian Zanjani F, Zinger S, Mihajlovic N, Ng G, Korsten H, With P (2018) Robust and semantic needle detection in 3D ultrasound using orthogonal-plane convolutional neural networks. Int J Comput Assist Radiol Surg 13(9):1321–1333CrossRef
19.
Zurück zum Zitat Mwikirize C, Nosher JL, Hacihaliloglu I (2018) Convolutional neural networks for real-time needle detection and localization in 2D ultrasound. Int J Comput Assist Radiol Surg 13(5):647–657CrossRef Mwikirize C, Nosher JL, Hacihaliloglu I (2018) Convolutional neural networks for real-time needle detection and localization in 2D ultrasound. Int J Comput Assist Radiol Surg 13(5):647–657CrossRef
20.
Zurück zum Zitat Mwikirize C, Nosher JL, Hacihaliloglu I (2019) Single shot needle localization in 2D ultrasound. In: MICCAI 2019. MICCAI LNCS, 11768, pp 637–645 Mwikirize C, Nosher JL, Hacihaliloglu I (2019) Single shot needle localization in 2D ultrasound. In: MICCAI 2019. MICCAI LNCS, 11768, pp 637–645
21.
Zurück zum Zitat Mwikirize C, Nosher JL, Hacihaliloglu I (2019) Learning needle tip localization from digital subtraction in 2D ultrasound. Int J Comput Assist Radiol Surg 14(6):1017–1026CrossRef Mwikirize C, Nosher JL, Hacihaliloglu I (2019) Learning needle tip localization from digital subtraction in 2D ultrasound. Int J Comput Assist Radiol Surg 14(6):1017–1026CrossRef
22.
Zurück zum Zitat Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 779–788 Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 779–788
Metadaten
Titel
Time-aware deep neural networks for needle tip localization in 2D ultrasound
verfasst von
Cosmas Mwikirize
Alvin B. Kimbowa
Sylvia Imanirakiza
Andrew Katumba
John L. Nosher
Ilker Hacihaliloglu
Publikationsdatum
11.04.2021
Verlag
Springer International Publishing
Erschienen in
International Journal of Computer Assisted Radiology and Surgery / Ausgabe 5/2021
Print ISSN: 1861-6410
Elektronische ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-021-02361-w

Weitere Artikel der Ausgabe 5/2021

International Journal of Computer Assisted Radiology and Surgery 5/2021 Zur Ausgabe

Screening-Mammografie offenbart erhöhtes Herz-Kreislauf-Risiko

26.04.2024 Mammografie Nachrichten

Routinemäßige Mammografien helfen, Brustkrebs frühzeitig zu erkennen. Anhand der Röntgenuntersuchung lassen sich aber auch kardiovaskuläre Risikopatientinnen identifizieren. Als zuverlässiger Anhaltspunkt gilt die Verkalkung der Brustarterien.

S3-Leitlinie zu Pankreaskrebs aktualisiert

23.04.2024 Pankreaskarzinom Nachrichten

Die Empfehlungen zur Therapie des Pankreaskarzinoms wurden um zwei Off-Label-Anwendungen erweitert. Und auch im Bereich der Früherkennung gibt es Aktualisierungen.

Fünf Dinge, die im Kindernotfall besser zu unterlassen sind

18.04.2024 Pädiatrische Notfallmedizin Nachrichten

Im Choosing-Wisely-Programm, das für die deutsche Initiative „Klug entscheiden“ Pate gestanden hat, sind erstmals Empfehlungen zum Umgang mit Notfällen von Kindern erschienen. Fünf Dinge gilt es demnach zu vermeiden.

„Nur wer sich gut aufgehoben fühlt, kann auch für Patientensicherheit sorgen“

13.04.2024 Klinik aktuell Kongressbericht

Die Teilnehmer eines Forums beim DGIM-Kongress waren sich einig: Fehler in der Medizin sind häufig in ungeeigneten Prozessen und mangelnder Kommunikation begründet. Gespräche mit Patienten und im Team können helfen.

Update Radiologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.