Skip to main content

Advertisement

SpringerLink
Log in

Workflow for Visualization of Neuroimaging Data with an Augmented Reality Device

  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

Commercial availability of three-dimensional (3D) augmented reality (AR) devices has increased interest in using this novel technology for visualizing neuroimaging data. Here, a technical workflow and algorithm for importing 3D surface-based segmentations derived from magnetic resonance imaging data into a head-mounted AR device is presented and illustrated on selected examples: the pial cortical surface of the human brain, fMRI BOLD maps, reconstructed white matter tracts, and a brain network of functional connectivity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Azuma RT: A survey of augmented reality. Presence Teleop Virt 6(4):355–385, 1997

    Article  Google Scholar 

  2. Milgram P, Kishino F: A taxonomy of mixed reality visual displays. IEIC Transactions on Information and Systems, E77D:12, 1321–29,1994

  3. Abe Y, Sato S, Kato K et al.: A novel 3D guidance system using augmented reality for percutaneous vertebroplasty. J Neurosurg Spine 19(4):492–501, 2013

    Article  PubMed  Google Scholar 

  4. Blackwell M, Morgan F, DiGioia, 3rd AM: Augmented reality and its future in orthopaedics. Clin Orthop Relat Res 354:111–122, 1998

    Article  Google Scholar 

  5. Kerner KF, et al.: Augmented reality for teaching endotracheal intubation: MR imaging to create anatomically correct models. AMIA Annu Symp Proc p. 888,2003

  6. Nicolau S et al.: Augmented reality in laparoscopic surgical oncology. Surg Oncol 20(3):189–201, 2011

    Article  PubMed  Google Scholar 

  7. Fritz J et al.: Augmented reality visualization using image overlay technology for MR-guided interventions: cadaveric bone biopsy at 1.5 T. Invest Radiol 48(6):464–470, 2013

    Article  PubMed  Google Scholar 

  8. Volonte F et al.: Augmented reality to the rescue of the minimally invasive surgeon. The usefulness of the interposition of stereoscopic images in the Da Vinci robotic console. Int J Med Robot 9(3):e34–e38, 2013

    Article  PubMed  Google Scholar 

  9. Markman A et al.: Augmented reality three-dimensional object visualization and recognition with axially distributed sensing. Opt Lett 41(2):297–300, 2016

    Article  PubMed  Google Scholar 

  10. Chinnock C: Virtual reality in surgery and medicine. Hosp Technol Ser 13(18):1–48, 1994

    CAS  PubMed  Google Scholar 

  11. Ota D et al.: Virtual reality in surgical education. Comput Biol Med 25(2):127–137, 1995

    Article  CAS  PubMed  Google Scholar 

  12. Olofsson J et al.: Advanced 3D-visualization, including virtual reality, distributed by PCs, in brain research, clinical radiology and education. Stud Health Technol Inform 50:357–358, 1998

    CAS  PubMed  Google Scholar 

  13. Webb G et al.: Virtual reality and interactive 3D as effective tools for medical training. Stud Health Technol Inform 94:392–394, 2003

    PubMed  Google Scholar 

  14. Farber M et al.: Virtual reality simulator for the training of lumbar punctures. Methods Inf Med 48(5):493–501, 2009

    Article  PubMed  Google Scholar 

  15. Clarke DB et al.: Virtual reality simulator: demonstrated use in neurosurgical oncology. Surg Innov 20(2):190–197, 2013

    Article  PubMed  Google Scholar 

  16. Mi SH et al.: A 3D virtual reality simulator for training of minimally invasive surgery. Conf Proc IEEE Eng Med Biol Soc 2014:349–352, 2014

    PubMed  Google Scholar 

  17. Khavari R et al.: Functional magnetic resonance imaging with concurrent urodynamic testing identifies brain structures involved in micturition cycle in patients with multiple sclerosis. J Urol 197:438–444, 2016

    Article  PubMed  PubMed Central  Google Scholar 

  18. Shy M et al.: Functional magnetic resonance imaging during urodynamic testing identifies brain structures initiating micturition. J Urol 192(4):1149–1154, 2014

    Article  PubMed  PubMed Central  Google Scholar 

  19. Bidgood, Jr WD, Horii SC: Introduction to the ACR-NEMA DICOM standard. Radiographics 12(2):345–355, 1992

    Article  PubMed  Google Scholar 

  20. John NW et al.: MedX3D: standards enabled desktop medical 3D. Stud Health Technol Inform 132:189–194, 2008

    CAS  PubMed  Google Scholar 

  21. Cox RW: AFNI: what a long strange trip it’s been. Neuroimage 62(2):743–747, 2012

    Article  PubMed  Google Scholar 

  22. Larobina M, Murino L: Medical image file formats. J Digit Imaging 27(2):200–206, 2014

    Article  PubMed  Google Scholar 

  23. Schneider CA, Rasband WS, Eliceiri KW: NIH image to ImageJ: 25 years of image analysis. Nat Methods 9(7):671–675, 2012

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Xie S et al.: DiffusionKit: a light one-stop solution for diffusion MRI data analysis. J Neurosci Methods 273:107–119, 2016

    Article  PubMed  Google Scholar 

  25. Karmonik C et al.: Music listening modulates functional connectivity and information flow in the human brain. Brain Connect, 2016. doi: 10.1089/brain.2016.0428

  26. Berlage T: Augmented-reality communication for diagnostic tasks in cardiology. IEEE Trans Inf Technol Biomed 2(3):169–173, 1998

    Article  CAS  PubMed  Google Scholar 

  27. Sato Y et al.: Image guidance of breast cancer surgery using 3-D ultrasound images and augmented reality visualization. IEEE Trans Med Imaging 17(5):681–693, 1998

    Article  CAS  PubMed  Google Scholar 

  28. Kawamata T et al.: Endoscopic augmented reality navigation system for endonasal transsphenoidal surgery to treat pituitary tumors: technical note. Neurosurgery 50(6):1393–1397, 2002

    PubMed  Google Scholar 

  29. Paul P, Fleig O, Jannin P: Augmented virtuality based on stereoscopic reconstruction in multimodal image-guided neurosurgery: methods and performance evaluation. IEEE Trans Med Imaging 24(11):1500–1511, 2005

    Article  PubMed  Google Scholar 

  30. Lukosch S, Billinghurst M, Alem L et al.: The effect of view independence in a collaborative AR system. Computer supported cooperative work. J Collab Comput 24(6):563–589, 2015

    Google Scholar 

  31. Lukosch S, Billinghurst M, Alem L et al.: Collaboration in augmented reality. Computer supported cooperative work. J Collab Comput 24(6):515–525, 2015

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. MRI core, Houston Methodist Hospital Research Institute, 6565 Fannin Street, Houston, TX, 77030, USA

    Christof Karmonik

  2. Department of Urology, Houston Methodist Hospital, Houston, TX, USA

    Timothy B. Boone & Rose Khavari

Authors
  1. Christof Karmonik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Timothy B. Boone
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rose Khavari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christof Karmonik.

Electronic supplementary material

(MP4 81.0 mb)

(MP4 87.9 mb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karmonik, C., Boone, T.B. & Khavari, R. Workflow for Visualization of Neuroimaging Data with an Augmented Reality Device. J Digit Imaging 31, 26–31 (2018). https://doi.org/10.1007/s10278-017-9991-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-017-9991-4

Keywords

Search

Navigation