Abstract
Our perception of the world's three-dimensional (3D) structure is critical for object recognition, navigation and planning actions. To accomplish this, the brain combines different types of visual information about depth structure, but at present, the neural architecture mediating this combination remains largely unknown. Here, we report neuroimaging correlates of human 3D shape perception from the combination of two depth cues. We measured fMRI responses while observers judged the 3D structure of two sequentially presented images of slanted planes defined by binocular disparity and perspective. We compared the behavioral and fMRI responses evoked by changes in one or both of the depth cues. fMRI responses in extrastriate areas (hMT+/V5 and lateral occipital complex), rather than responses in early retinotopic areas, reflected differences in perceived 3D shape, suggesting 'combined-cue' representations in higher visual areas. These findings provide insight into the neural circuits engaged when the human brain combines different information sources for unified 3D visual perception.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Rent or buy this article
Prices vary by article type
from$1.95
to$39.95
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Landy, M.S., Maloney, L.T., Johnston, E.B. & Young, M. Measurement and modeling of depth cue combination – in defense of weak fusion. Vision Res. 35, 389–412 (1995).
Clark, J.J. & Yuille, A.L. Data fusion for sensory information processing systems (Kluwer Academic, Boston, 1990).
Hillis, J.M., Ernst, M.O., Banks, M.S. & Landy, M.S. Combining sensory information: mandatory fusion within, but not between, senses. Science 298, 1627–1630 (2002).
Nguyenkim, J.D. & DeAngelis, G.C. Disparity-based coding of three-dimensional surface orientation by macaque middle temporal neurons. J. Neurosci. 23, 7117–7128 (2003).
Backus, B.T., Fleet, D.J., Parker, A.J. & Heeger, D.J. Human cortical activity correlates with stereoscopic depth perception. J. Neurophysiol. 86, 2054–2068 (2001).
Tsao, D.Y. et al. Stereopsis activates V3A and caudal intraparietal areas in macaques and humans. Neuron 39, 555–568 (2003).
Tsutsui, K., Sakata, H., Naganuma, T. & Taira, M. Neural correlates for perception of 3D surface orientation from texture gradient. Science 298, 409–412 (2002).
Liu, Y., Vogels, R. & Orban, G.A. Convergence of depth from texture and depth from disparity in macaque inferior temporal cortex. J. Neurosci. 24, 3795–3800 (2004).
Grill-Spector, K. & Malach, R. fMR-adaptation: a tool for studying the functional properties of human cortical neurons. Acta Psychol. (Amst.) 107, 293–321 (2001).
Kourtzi, Z. & Kanwisher, N. Representation of perceived object shape by the human lateral occipital complex. Science 293, 1506–1509 (2001).
Knill, D.C. & Saunders, J.A. Do humans optimally integrate stereo and texture information for judgments of surface slant? Vision Res. 43, 2539–2558 (2003).
Grill-Spector, K., Kourtzi, Z. & Kanwisher, N. The lateral occipital complex and its role in object recognition. Vision Res. 41, 1409–1422 (2001).
Zeki, S. et al. A direct demonstration of functional specialization in human visual cortex. J. Neurosci. 11, 641–649 (1991).
Tootell, R.B. et al. Functional analysis of human MT and related visual cortical areas using magnetic resonance imaging. J. Neurosci. 15, 3215–3230 (1995).
Kourtzi, Z., Buelthoff, H.H., Erb, M. & Grodd, W. Object-selective responses in the human motion area MT/MST. Nat. Neurosci. 5, 17–18 (2002).
Orban, G.A. et al. Similarities and differences in motion processing between the human and macaque brain: evidence from fMRI. Neuropsychologia 41, 1757–1768 (2003).
Janssen, P., Vogels, R. & Orban, G.A. Selectivity for 3D shape that reveals distinct areas within macaque inferior temporal cortex. Science 288, 2054–2056 (2000).
Neri, P., Bridge, H. & Heeger, D.J. Stereoscopic processing of absolute and relative disparity in human visual cortex. J. Neurophysiol. 92, 1880–1891 (2004).
Ernst, M.O. & Banks, M.S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 429–433 (2002).
Enright, J.T. Art and the oculomotor system: perspective illustrations evoke vergence changes. Perception 16, 731–746 (1987).
Ress, D., Backus, B.T. & Heeger, D.J. Activity in primary visual cortex predicts performance in a visual detection task. Nat. Neurosci. 3, 940–945 (2000).
Thomas, O.M., Cumming, B.G. & Parker, A.J. A specialization for relative disparity in V2. Nat. Neurosci. 5, 472–478 (2002).
Cumming, B.G. & Parker, A.J. Responses of primary visual cortical neurons to binocular disparity without depth perception. Nature 389, 280–283 (1997).
Watanabe, M., Tanaka, H., Uka, T. & Fujita, I. Disparity-selective neurons in area V4 of macaque monkeys. J. Neurophysiol. 87, 1960–1973 (2002).
Nienborg, H., Bridge, H., Parker, A.J. & Cumming, B.G. Receptive field size in V1 neurons limits acuity for perceiving disparity modulation. J. Neurosci. 24, 2065–2076 (2004).
Taira, M., Tsutsui, K.I., Jiang, M., Yara, K. & Sakata, H. Parietal neurons represent surface orientation from the gradient of binocular disparity. J. Neurophysiol. 83, 3140–3146 (2000).
Shikata, E. et al. Surface orientation discrimination activates caudal and anterior intraparietal sulcus in humans: an event-related fMRI study. J. Neurophysiol. 85, 1309–1314 (2001).
Poggio, G.F., Gonzalez, F. & Krause, F. Stereoscopic mechanisms in monkey visual cortex: binocular correlation and disparity selectivity. J. Neurosci. 8, 4531–4550 (1988).
von der Heydt, R., Zhou, H. & Friedman, H.S. Representation of stereoscopic edges in monkey visual cortex. Vision Res. 40, 1955–1967 (2000).
Hinkle, D.A. & Connor, C.E. Three-dimensional orientation tuning in macaque area V4. Nat. Neurosci. 5, 665–670 (2002).
Bakin, J.S., Nakayama, K. & Gilbert, C.D. Visual responses in monkey areas V1 and V2 to three-dimensional surface configurations. J. Neurosci. 20, 8188–8198 (2000).
Tanaka, H., Uka, T., Yoshiyama, K., Kato, M. & Fujita, I. Processing of shape defined by disparity in monkey inferior temporal cortex. J. Neurophysiol. 85, 735–744 (2001).
Uka, T., Tanaka, H., Yoshiyama, K., Kato, M. & Fujita, I. Disparity selectivity of neurons in monkey inferior temporal cortex. J. Neurophysiol. 84, 120–132 (2000).
Janssen, P., Vogels, R., Liu, Y. & Orban, G.A. Macaque inferior temporal neurons are selective for three-dimensional boundaries and surfaces. J. Neurosci. 21, 9419–9429 (2001).
Gilaie-Dotan, S., Ullman, S., Kushnir, T. & Malach, R. Shape-selective stereo processing in human object-related visual areas. Hum. Brain Mapp. 15, 67–79 (2002).
Bradley, D.C., Chang, G.C. & Andersen, R.A. Encoding of three-dimensional structure-from-motion by primate area MT neurons. Nature 392, 714–717 (1998).
Palanca, B.J. & DeAngelis, G.C. Macaque middle temporal neurons signal depth in the absence of motion. J. Neurosci. 23, 7647–7658 (2003).
Dodd, J.V., Krug, K., Cumming, B.G. & Parker, A.J. Perceptually bistable three-dimensional figures evoke high choice probabilities in cortical area MT. J. Neurosci. 21, 4809–4821 (2001).
Sary, G., Vogels, R. & Orban, G.A. Cue-invariant shape selectivity of macaque inferior temporal neurons. Science 260, 995–997 (1993).
Grill-Spector, K., Kushnir, T., Edelman, S., Itzchak, Y. & Malach, R. Cue-invariant activation in object-related areas of the human occipital lobe. Neuron 21, 191–202 (1998).
Albright, T.D. Cortical processing of visual motion. Rev. Oculomot. Res. 5, 177–201 (1993).
Treue, S. & Andersen, R.A. Neural responses to velocity gradients in macaque cortical area MT. Vis. Neurosci. 13, 797–804 (1996).
Murray, S.O., Olshausen, B.A. & Woods, D.L. Processing shape, motion and three-dimensional shape-from-motion in the human cortex. Cereb. Cortex 13, 508–516 (2003).
Paradis, A.L. et al. Visual perception of motion and 3-D structure from motion: an fMRI study. Cereb. Cortex 10, 772–783 (2000).
Taira, M., Nose, I., Inoue, K. & Tsutsui, K. Cortical areas related to attention to 3D surface structures based on shading: an fMRI study. Neuroimage 14, 959–966 (2001).
Sereno, M.E., Trinath, T., Augath, M. & Logothetis, N.K. Three-dimensional shape representation in monkey cortex. Neuron 33, 635–652 (2002).
van Ee, R., van Dam, L.C. & Erkelens, C.J. Bi-stability in perceived slant when binocular disparity and monocular perspective specify different slants. J. Vis. 2, 597–607 (2002).
Backus, B.T. in Advances in Neural Information Processing Systems 14 (eds. Dietterich, T.G., Becker, S. & Ghahramani, Z.) (MIT Press, Cambridge, Massachusetts, 2002).
Todd, J.T. The visual perception of 3D shape. Trends Cogn. Sci. 8, 115–121 (2004).
DeAngelis, G.C. & Newsome, W.T. Organization of disparity-selective neurons in macaque area MT. J. Neurosci. 19, 1398–1415 (1999).
Acknowledgements
Preliminary reports of this work were presented at the VisionSciences Society's 2003 meeting and at the Society for Neuroscience's 2003 meeting. Thanks to S. Maier and J. Lam for help with data collection, and N. Logothetis, N. Kanwisher, J. Harris, S. McDonald, M. Ernst and R. Fleming for helpful discussions and comments. Thanks also to R. van Ee for advice on stimulus generation. Supported by an Alexander von Humboldt Fellowship to A.E.W., the Max-Planck Society and DFG grant TH812/1-1.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Supplementary information
Supplementary Fig. 1
fMRI responses separated to account for between-subjects variability in cue weights. (PDF 79 kb)
Supplementary Fig. 2
Eye movement controls. (PDF 367 kb)
Supplementary Fig. 3
Results from an experiment on consistent-cue stimuli. (PDF 204 kb)
Supplementary Fig. 4
Additional example flatmaps showing examined ROIs. (PDF 737 kb)
Supplementary Fig. 5
Fits to the time course of the fMRI responses in the examined ROIs. (PDF 377 kb)
Supplementary Fig. 6
Mean peak fMRI response across the visual areas when individual reference stimuli are presented alone. (PDF 191 kb)
Supplementary Table 1
Measurement of psychophysical behavior of each observer. (PDF 515 kb)
Rights and permissions
About this article
Cite this article
Welchman, A., Deubelius, A., Conrad, V. et al. 3D shape perception from combined depth cues in human visual cortex. Nat Neurosci 8, 820–827 (2005). https://doi.org/10.1038/nn1461
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nn1461
This article is cited by
-
Cortical Deficits are Correlated with Impaired Stereopsis in Patients with Strabismus
Neuroscience Bulletin (2023)
-
Artificial Intelligence, 3D Documentation, and Rock Art—Approaching and Reflecting on the Automation of Identification and Classification of Rock Art Images
Journal of Archaeological Method and Theory (2022)
-
Human primary visual cortex shows larger population receptive fields for binocular disparity-defined stimuli
Brain Structure and Function (2021)
-
Lighting-from-above prior in biological motion perception
Scientific Reports (2018)
-
Burge on perception and sensation
Synthese (2016)