Research report
Distinctions between manipulation and function knowledge of objects: evidence from functional magnetic resonance imaging

https://doi.org/10.1016/j.cogbrainres.2004.11.001Get rights and content

Abstract

A prominent account of conceptual knowledge proposes that information is distributed over visual, tactile, auditory, motor and verbal-declarative attribute domains to the degree to which these features were activated when the knowledge was acquired [D.A. Allport, Distributed memory, modular subsystems and dysphagia, In: S.K. Newman, R. Epstein (Eds.), Current perspectives in dysphagia, Churchill Livingstone, Edinburgh, 1985, pp. 32–60]. A corollary is that when drawing upon this knowledge (e.g., to answer questions), particular aspects of this distributed information is re-activated as a function of the requirements of the task at hand [L.J. Buxbaum, E.M. Saffran, Knowledge of object manipulation and object function: dissociations in apraxic and non-apraxic subjects. Brain and Language, 82 (2002) 179–199; L.J. Buxbaum, T. Veramonti, M.F. Schwartz, Function and manipulation tool knowledge in apraxia: knowing ‘what for’ but not ‘how’, Neurocase, 6 (2000) 83–97; W. Simmons, L. Barsalou, The similarity-in-topography principle: Reconciling theories of conceptual deficits, Cognitive Neuropsychology, 20 (2003) 451–486]. This account predicts that answering questions about object manipulation should activate brain regions previously identified as components of the distributed sensory-motor system involved in object use, whereas answering questions about object function (that is, the purpose that it serves) should activate regions identified as components of the systems supporting verbal-declarative features. These predictions were tested in a functional magnetic resonance imaging (fMRI) study in which 15 participants viewed picture or word pairs denoting manipulable objects and determined whether the objects are manipulated similarly (M condition) or serve the same function (F condition). Significantly greater and more extensive activations in the left inferior parietal lobe bordering the intraparietal sulcus were seen in the M condition with pictures and, to a lesser degree, words. These findings are consistent with the known role of this region in skilled object use [K.M. Heilman, L.J. Gonzalez Rothi, Apraxia, In: K.M. Heilman, E. Valenstein (Eds.), Clinical Neuropsychology, Oxford University Press, New York, 1993, pp. 141–150] as well as previous fMRI results [M. Kellenbach, M. Brett, K. Patterson, Actions speak louder than functions: the importance of manipulability and action in tool representation, Journal of Cognitive Neuroscience, 15 (2003) 30–46] and behavioral findings in brain-lesion patients [L.J. Buxbaum, E.M. Saffran, Knowledge of object manipulation and object function: dissociations in apraxic and non-apraxic subjects, Brain and Language, 82 (2002) 179–199]. No brain regions were significantly more activated in the F than M condition. These data suggest that brain regions specialized for sensory-motor function are a critical component of distributed representations of manipulable objects.

Introduction

A growing body of evidence indicates that representations of man-made objects (hereafter, artifacts) can be distinguished from representations of living things (see Refs. [11], [16], [48] for recent reviews). The reasons for these dissociations remain unclear. One prominent account suggests that animals and artifacts differ in the underlying format of their representations. Based on the observation that exemplars of a given artifact—different telephones, for example—may differ in their size, shape, and color, yet still share a common function, it has been suggested that functional properties play a central role in the distinctions among artifacts, whereas visual properties and other sensory information distinguish living things [52], [53]; see also [18], [20], [47]. This account, however, does not explain several findings, including preservation of functional feature knowledge in patients with artifact impairments (e.g., Ref. [28]), sparing of knowledge of large artifacts with clear functions (buildings, vehicles) in patients with deficits for smaller manipulable artifacts [52], or co-occurrence of deficits in body-part knowledge and artifact knowledge [41], [52] (and see Ref. [11]). An alternative account having the potential to explain these patterns is the “sensory-motor” (SM) account. The SM account is perhaps most clearly articulated in a distributed model of the semantic system proposed by Allport [3] (see also Refs. [52], [53]). This model proposes that living and non-living things alike are represented in terms of auto-associated auditory, olfactory, visual and tactile-motor attribute domains as a function of the degree to which these forms of information were activated when the knowledge was acquired. On some related accounts, the distributed semantic system includes verbal-propositional elements as well [7]. On this model, the observed dissociations between living things and artifacts (and between large and small artifacts) might reflect the degree to which the objects under consideration have been held, moved and manipulated.

Consistent with this proposal, there is growing evidence that function and manipulation information are dissociable forms of object knowledge associated with different lesion loci, and that manipulation knowledge may be closely tied to overall knowledge of artifacts. Deficits in tool function knowledge in the face of relatively intact manipulation knowledge have occurred in patients with temporal lobe lesions due to semantic dementia or herpes encephalitis [7], [30], [45]. Deficits in manipulation but not function knowledge have occurred in apraxic patients with left frontoparietal lesions affecting sensory-motor association cortex [8]. In a recent group study [6], we tested left hemisphere stroke patients with measures of limb apraxia (gesture pantomime), tests of tool, animal and body part knowledge and knowledge of the function and manipulation of manipulable artifacts. The function and manipulation knowledge test required a judgment of synonymy about a triplet of items. In the function condition, for example, subjects were shown a record player, radio and telephone; only the first two objects are alike in function and all are held and used differently. In the manipulation condition, subjects were shown, for instance, a typewriter, piano and stove; the first two are both ‘tapped’ with the fingertips, whereas a stove is not, and all serve a different function. There were picture and word versions of the test. Patients with relative deficits in tool knowledge tended to be apraxic, impaired in manipulation (but not function) knowledge with both picture and word stimuli, deficient in body part knowledge and to have left inferior parietal lesions. The coherence of this pattern supports the contention that manipulation knowledge subserved by sensory-motor association cortices may play a prominent role in the representations of tools (but see Ref. [31] for objections).

Consistent with this possibility, cortical activations reported in neuroimaging studies of artifact or tool representations include the left posterior parietal lobe and/or intraparietal sulcus (IPS), left ventral premotor cortex (PMv) of the frontal lobe and left posterior middle temporal gyrus (PMTG) [12], [21], [24], [32] (see also Refs. [16], [27] for review). All three regions (albeit not necessarily lateralized to the left) have been associated with action representations in prior neurophysiological studies of the monkey [4], [33], [39], [43].

Several recent imaging studies have directly tested the hypothesis that activations associated with manipulable artifacts reflect action-related processes. Gerlach et al. [22] demonstrated using fMRI that categorizing pictures as natural or man-made activated PMv for manipulable objects (fruit/vegetables and clothing) more than animals and non-manipulable objects. Phillips et al. [37] showed PET activations in the left PMTG with action decisions about manipulable objects. In keeping with the findings of Gerlach et al. [22], this pattern was observed even for small manipulable living things (i.e., fruit), suggesting that the activation was not attributable to an effect of semantic category per se. In a recent PET study by Kellenbach, Brett and Patterson [27], participants were shown pictures of manipulable and non-manipulable objects, and asked to make judgments about their function (e.g., is this used to attach objects together?) and, for manipulable objects, the manner in which they are manipulated (e.g., is this used with a swinging motion?). There were common activations in left PMv and left PMTG in all object conditions relative to a control condition, but these regions were also relatively more activated by manipulable objects (irrespective of task) than non-manipulable objects. This is consistent with the possibility that these regions are involved in processing action-related information about manipulable objects. Additionally, there was highly significant activation of left posterior parietal lobe in the region of the IPS when subjects responded to questions about actions associated with manipulable objects. The specificity of these activations led the investigators to suggest that this region may be selectively involved with explicit retrieval of action knowledge.

A question that is not addressed by the study of Kellenbach et al. [27] is whether the results obtained are specific to pictures of objects. This is critical given behavioral evidence that pictures may provide more rapid access to action knowledge than do words [40], [42], and functional imaging evidence that pictures and words may be processed differently, particularly in the domains of function and action knowledge. Using PET, Vandenberghe et al. [51] identified a large distributed system associated with processing the functional associates of both pictures and words, along with unique activations in left posterior inferior temporal sulcus for picture stimuli and left superior temporal sulcus, left anterior middle temporal gyrus and left inferior frontal gyrus for words. In a recent PET study, Phillips et al. [37] also identified a large left inferior frontal and anterior temporal region commonly activated for action decisions (e.g., “do you use this with a twisting motion?”) for both pictures and words, as well as left inferior frontal, left anterior fusiform, and left anterior temporal regions that were more activated for such decisions with words than pictures. They concluded that the reduction in semantic activation in the picture condition resulted from facilitation of direct and automatic access to action by the visual-motor affordances present in objects.

On the basis of these considerations, we assessed the neural correlates of function and manipulation knowledge using both picture and word formats with a variant of the synonymy judgment task we had developed previously [6]. The inclusion of word stimuli permits us to address the critical question of whether activations observed with pictures are automatically driven by object affordances (i.e., shape and size information compatible with, and linked to, particular motor actions, e.g., as the handle of a hammer is compatible with a prehensile hand posture [23]). If activations are entirely affordance-related, that is, automatically driven by visual shape and size information, then they should not be observed with word stimuli. Subjects also performed two baseline tasks designed to parcel out visual-perceptual and motor response processes. We aimed to address the following general hypotheses: (1) function and manipulation knowledge of objects are associated with partially overlapping, but partially distinct patterns of neural activation; (2) manipulation representations are not solely triggered automatically by object affordances, but in part reflect object-specific, abstract, modality-independent processes; and (3) the brain is more responsive to the actions associated with objects than to their functions. More specifically, the study was designed to test several predictions. First, consistent with the previous literature, we expected overlapping regions of activation in dorsolateral and ventrolateral prefrontal and premotor cortex, MPTG, and IPL when subjects answered questions about the manipulation (M) and function (F) of manipulable artifacts. Second, we expected that F and M conditions would also be associated with distinct patterns of activation. Based on previous observations, we expected that the manipulation (M) condition would disproportionately activate the left IPL; an expectation from the neuropsychological literature was that the function (F) condition would give rise to relative activations in the temporal lobe. Third, we expected to see many of the same brain regions activated with word and picture stimuli. Finally, given the results of Phillips et al. [37] and Vandenberghe et al. [51], we also expected that we might find specific differences between pictures and words in the function condition (left posterior inferior temporal sulcus, left superior temporal sulcus), manipulation condition (left anterior fusiform gyrus) or both (left inferior frontal, left anterior middle temporal gyri).

Section snippets

Participants

Twenty-two healthy subjects were recruited from the University of Pennsylvania community and scanned. Three subjects were not included in data analysis because of incomplete behavioral data, two because of incomplete fMRI data and two because their accuracy scores were >2 S.D. below the average, leaving 15 participants (8 females, 7 males). Subjects were aged 21–33 (mean, 23.1), spoke English as a first language, were right handed and had normal or corrected to normal vision. All gave informed

Reaction time

Summary data are presented in Table 1. As there was no principled reason to think that the baseline conditions for the function (F) and manipulation (M) conditions were different (i.e., both consisted of scrambled, unrecognizable pictures or words), baseline data from the F and M conditions were collapsed into two conditions: (B)aseline pictures and B words. An analysis of variance (ANOVA) was performed on the reaction time (RT) data. This analysis, which included the four primary conditions of

Discussion

In this study, we addressed four major predictions generated by the SM theory regarding the semantic representations of manipulable artifacts. First, consistent with the previous literature, we expected overlapping regions of activation in dorsolateral and ventrolateral prefrontal and premotor cortex, MPTG and IPL in F and M conditions. Second, we expected that F and M conditions would also be associated with distinct patterns of activation. Specifically, we expected that the manipulation (M)

Acknowledgments

Supported by NIH grants NS36387, NS37920, DC02754, NS045839 and HD39621.

References (55)

  • S. Thompson-Schill

    Neuroimaging studies of semantic memory: inferring “how” from “where”

    Neuropsychologia

    (2003)
  • C. Whatmough et al.

    Dissociable brain regions process object meaning and object structure during picture naming

    Neuropsychologia

    (2002)
  • D.A. Allport

    Distributed memory, modular subsystems and dysphagia

  • R.A. Andersen

    Coordinate Transformations and Motor Planning in Posterior Parietal Cortex

    (1995)
  • D. Bub et al.

    Gesturing and naming: the use of functional knowledge in object identification

    Psychological Science

    (2003)
  • L.J. Buxbaum et al.

    The role of semantic memory in object use

    Cognitive Neuropsychology

    (1997)
  • L.J. Buxbaum et al.

    Function and manipulation tool knowledge in apraxia: knowing ‘what for’ but not ‘how’

    Neurocase

    (2000)
  • L.J. Buxbaum, S.H. Johnson-Frey, M. Bartlett-Williams, Deficient internal models for planning hand-object interactions...
  • E. Capitani et al.

    What are the facts of semantic category-specific deficits? A critical review of the clinical evidence

    Cognitive Neuropsychology

    (2003)
  • D.L. Collins

    3D Model-Based Segmentation of Individual Brain Structures From Magnetic Resonance Imaging Data

    (1994)
  • M. Corbetta et al.

    Control of goal-directed and stimulus-driven attention in the brain

    Nature Reviews Neuroscience

    (2002)
  • L. Craighero et al.

    Visuomotor priming

    Visual Cognition

    (1998)
  • M.J. Farah et al.

    A computational model of semantic memory impairment: modality specificity and emergent category specificity

    Journal of Experimental Psychology, General

    (1991)
  • K. Friston et al.

    Spatial registration and normalization of images

    Human Brain Mapping

    (1995)
  • P. Garrard et al.

    Category-specific semantic loss in dementia of Alzheimer's type

    Brain

    (1998)
  • C. Gerlach et al.

    When action turns into words. Activation of motor-based knowledge during categorization of manipulable objects

    Journal of Cognitive Neuroscience

    (2002)
  • J.J. Gibson

    The theory of affordances

  • Cited by (227)

    • Visuomotor control in the healthy and damaged brain

      2021, Encyclopedia of Behavioral Neuroscience: Second Edition
    View all citing articles on Scopus
    View full text