Hierarchical coding of characters in the ventral and dorsal visual streams of Chinese language processing
Introduction
fMRI studies using a variety of different task paradigms have reported success earlier in localizing the brain activations in Chinese language processing (Chee et al., 1999a, Chee et al., 1999c, Chee et al., 2000, Tan et al., 2000, Chee et al., 2001, Tan et al., 2001a, Tan et al., 2001b, Chen et al., 2002, Kuo et al., 2003, Matthews et al., 2003, Kuo et al., 2004). The brain activity in the major neural circuits in the prefrontal cortex, dorsal visual stream and ventral visual stream (Dejerine, 1892, Geschwind, 1965a, Geschwind, 1965b, Petersen et al., 1990, Price, 2000, Jobard et al., 2003) for Chinese character reading did not show much difference from the reading of an alphabetic language like English, in spite of the marked difference between the logographic system of Chinese and alphabetic language. However, the brain activations related to the difference of the orthographic structure in the input stimuli have yet to be investigated in details in Chinese language studies.
Much evidence has been provided that neurons in humans are tuned to become selectively efficient at processing language inputs that are encountered more frequently (Kuo et al., 2003, Baker et al., 2007). In language processing, reading of a printed word is first processed in the striate cortex of the occipital lobe. The lateral extrastriate and the left occipitotemporal cortices in the ventral visual stream form a linguistically organized system for pattern-based process of word identification (Poldrack et al., 1998, Buchel et al., 1999, Price, 2000, Jobard et al., 2003). A patch of cortex lateral to the midportion of the fusiform gyrus had been proposed as the ‘visual word form area’ (Cohen et al., 2000). Neuronal correlates underpinning the orthographic and phonological computations for alphabetic words have been studied by tasks with different demands, with stimuli of varying orthographic legality/regularity, that is, words, pseudowords, consonant letter strings and false fonts (Petersen et al., 1990, Price et al., 1996, Fiez et al., 1999, Hagoort et al., 1999, Mechelli et al., 2000, Paulesu et al., 2000, Cohen et al., 2002, Dehaene et al., 2002, Mechelli et al., 2003). The argument of left fusiform gyrus processes as ‘word forms’, was further strengthened by the spatially graded response correlated with the degree of orthographic structure in a letter string by varying the frequency of occurrence in the language of letters and short letter sequences within the string. A ‘gradient of selectivity’ was observed through the entire span of the left occipitotemporal cortex based on the local combination detector (LCD) model first proposed by Dehaene et al. (2005), with activation becoming more selective for higher-level stimuli towards the anterior fusiform region (Vinckier et al., 2007). The LCD model starts with the hypothesis that word recognition obeys the same principles that govern the organization of the primate ventral visual system in recognizing complex and simple objects (Booth and Rolls, 1998, Rolls, 2000), where the hierarchy of word recognition is achieved by pooling activation from populations of neurons, in parallel to a progressive spatially oriented increase in the complexity of the neurons' preferred features (Dehaene et al., 2005). Apart from the left occipitotemporal cortex, other language processing areas including Broca's area and the insula also demonstrated such a spatial gradient of responses in French by Vinckier et al. (2007). Although the ‘gradient of selectivity’ proposed by Vinckier et al. is in parallel with the ‘word superiority effect’ in English proposed by Binder et al. (2006), Binder et al. showed that this spatially graded response was specific to the left lateral fusiform gyrus (Binder et al., 2006). Considering the difference of morphomes in logographic and alphabetic language systems, unlike the linear arrangement of alphabetic words, each Chinese character consists of strokes or radicals. All Chinese characters are in square configuration. The radical is found in conventional position in characters and substantially affects the search efficiency in character recognition (Yeh and Li, 2002). Some initial answer of the hierarchical coding of Chinese characters at the occipitotemporal cortex has appeared in two similar reports (Kuo et al., 2004, Liu et al., 2008). We are raising two questions in this manuscript. First, can we find a more detailed map of the hierarchical coding of Chinese character reading, delineating better the spatial gradient of increasing selectivity to Chinese characters at the occipitotemporal cortex, similar to what was reported for alphabetic language? Secondly, would such a hierarchical spatial gradient also be found in other language processing areas outside of the occipitotemporal cortex, given that ample evidence has suggested that reading is supported by distributed and interconnected neural networks (Fiez and Petersen, 1998)?
In the two cited previous works (Kuo et al., 2004, Liu et al., 2008), Kuo et al. reported different levels of Chinese orthographic and phonological processing using homophone judgment and three physical judgments of Chinese characters, pseudo-characters and Korean-like nonsense figures (Kuo et al., 2004). They showed that the left occipitotemporal region, the left dorsal stream from occipital to parietal cortex, and the right middle frontal gyrus constituted a network for orthographic processing, while the left premotor gyrus, left middle/inferior frontal gyrus, supplementary motor area and the left temporoparietal region worked in phonological processing. They showed decreasing signal change in BOLD magnitude from Chinese characters to Korean characters but did not, however, clearly map the size and extent of the spatial gradient of character type selectivity at the occipitotemporal cortex. This can partially be a consequence of their statistical strategies, where multiple comparison tests were absent. Results of BOLD signal intensity variation were similarly reported by Liu et al. (2008), with a listing of Talairach coordinates, again without a detailed mapping of the size and extent of the spatial gradient of character type selectivity. Liu et al. presented real Chinese characters, pseudo-characters, artificial characters and checkerboard for implicit linguistic processing. It should be mentioned that there is also a body of work that explored the training effect of learning Korean characters as unfamiliar foreign writing on the BOLD responses of the fusiform cortex (Xue et al., 2006, Xue and Poldrack, 2007). However, visual expertise acquired with training is different from the focus of our study.
In this manuscript, we studied in finer detail the spatial mapping of the neuronal tuning of Chinese characters at the occipitotemporal cortex. In addition, we also investigated and looked for new regions of graded selectivity to Chinese characters outside the occipitotemporal cortex. In our investigation, the active tasks we chose for our subjects share similarity as well as difference with previous literature. We used tasks corresponding to three different orthographic levels of decreasing familiarity: Chinese synonyms/non-synonyms (SYN), identical/non-identical pseudo-characters (PSE) and identical/non-identical Korean characters (KOR). Owing to the difference of the logographic and alphabetic systems, an analogy of letter strings with varying bigrams and trigrams in pseudo-characters may not be reproducible in a language study of the continuum of varying orthographic structure. We studied whether Chinese language processing areas which exhibit the neuronal tuning in response to the characters of different orthographic structure are similar to those responding to word strings in alphabetic systems with a spatial gradient of selectivity. We expected to show the effect of orthographic familiarity on the brain responses to the stimuli with increasing structural similarity in the order going from Korean characters to pseudo-characters and to real Chinese characters. Obviously we could not eliminate the contribution of phonological processing and semantic processing whenever real Chinese characters were involved in the comparisons.
Section snippets
Participants
Twenty-two right-handed healthy adults (12 males, 10 females, aged from 19 to 30 years) were included. They were recruited on the campus of The Hong Kong Polytechnic University. A questionnaire adapted from Edinburgh Inventory (Oldfield, 1971) was administered to each subject to determine the degree of handedness. All the subjects were right-handed native Cantonese speakers and did not know Korean. None of them had known or suspected neurological disease. MRI scanning was then performed on each
Results
Significant positive activations in the prefrontal cortex, bilateral ventral and dorsal visual streams were demonstrated in the processing of SYN, PSE and KOR relative to FIX. Negative BOLD responses were also noted. Throughout this manuscript, “negative BOLD responses” and “deactivation” are mutually exchangeable terms as both have been used in the literature and both refer to negative going BOLD signal in comparison with the baseline signal of a control condition. The signal changes in the
Discussion
We aimed to investigate the brain activation pattern of neuronal tuning in perceiving Chinese synonyms/non-synonyms, identical/non-identical pseudo-characters and identical/non-identical Korean characters, stimuli which represent a decreasing familiarity in orthographic structure and semantic elements. In addition to studying the left occipitotemporal cortex, other language processing areas in the ventral visual stream, dorsal visual stream and prefrontal cortex were also examined for the
Conclusion
We observed a posterior-to-anterior gradient of character type selectivity with increasing sensitivity from Korean characters to pseudo-characters at the left occipitotemporal cortex in the ventral visual stream. At the left and right intraparietal sulci of the dorsal visual stream, a medial-to-lateral gradient of character type selectivity with increasing sensitivity to real Chinese characters was also observed. The results suggest that the left occipitotemporal cortex and left intraparietal
Acknowledgments
This research was supported by the Hong Kong Research Grants Council (PolyU 5403/04M) and The Hong Kong Polytechnic University. We thank the radiographers at the United Christian Hospital and the Queen Elizabeth Hospital for their assistance in data collection.
References (80)
- et al.
A parametric fMRI study of overt and covert shifts of visuospatial attention
NeuroImage
(2001) - et al.
Tuning of the human left fusiform gyrus to sublexical orthographic structure
NeuroImage
(2006) - et al.
Processing of visually presented sentences in Mandarin and English studied with fMRI
Neuron
(1999) - et al.
Overlap and dissociation of semantic processing of Chinese characters, English words, and pictures: evidence from fMRI
NeuroImage
(2000) - et al.
Relative language proficiency modulates BOLD signal change when bilinguals perform semantic judgments. Blood oxygen level dependent
NeuroImage
(2001) - et al.
Top-down and bottom-up attention to memory: a hypothesis (AtoM) on the role of the posterior parietal cortex in memory retrieval
Neuropsychologia
(2008) AFNI: software for analysis and visualization of functional magnetic resonance neuroimages
Comput. Biomed. Res.
(1996)- et al.
The neural code for written words: a proposal
Trends Cogn. Sci.
(2005) - et al.
Effects of lexicality, frequency, and spelling-to-sound consistency on the functional anatomy of reading
Neuron
(1999) - et al.
Cortical surface-based analysis. II: Inflation, flattening, and a surface-based coordinate system
NeuroImage
(1999)
Eccentricity bias as an organizing principle for human high-order object areas
Neuron
Evaluation of the dual route theory of reading: a metanalysis of 35 neuroimaging studies
NeuroImage
The reviewing of object files: object-specific integration of information
Cogn. Psychol.
The subcortical role of language processing. High level linguistic features such as ambiguity-resolution and the human brain; an fMRI study
NeuroImage
Frequency effects of Chinese character processing in the brain: an event-related fMRI study
NeuroImage
Orthographic and phonological processing of Chinese characters: an fMRI study
NeuroImage
Indexing and the object concept: developing ‘what’ and ‘where’ systems
Trends Cogn. Sci.
The visual word form area: evidence from an fMRI study of implicit processing of Chinese characters
NeuroImage
Mapping the functional connectivity of anterior cingulate cortex
NeuroImage
Interrupting the “stream of consciousness”: an fMRI investigation
NeuroImage
Competition for priority in processing increases prefrontal cortex's involvement in top-down control: an event-related fMRI study of the Stroop task
Brain Res. Cogn. Brain Res.
The assessment and analysis of handedness: the Edinburgh inventory
Neuropsychologia
Functions of the primate temporal lobe cortical visual areas in invariant visual object and face recognition
Neuron
Functional MR imaging in assessment of language dominance in epileptic patients
NeuroImage
The neural system underlying Chinese logograph reading
NeuroImage
‘What’ and ‘where’ in the human brain
Curr. Opin. Neurobiol.
Hierarchical coding of letter strings in the ventral stream: dissecting the inner organization of the visual word-form system
Neuron
Language experience shapes fusiform activation when processing a logographic artificial language: an fMRI training study
NeuroImage
Visual word processing and experiential origins of functional selectivity in human extrastriate cortex
Proc. Natl. Acad. Sci. U. S. A.
Language dominance determined by whole brain functional MRI in patients with brain lesions
Neurology
Function of the left planum temporale in auditory and linguistic processing
Brain
Determination of language dominance using functional MRI: a comparison with the Wada test
Neurology
Conceptual processing during the conscious resting state. A functional MRI study
J. Cogn. Neurosci.
Neural correlates of lexical access during visual word recognition
J. Cogn. Neurosci.
View-invariant representations of familiar objects by neurons in the inferior temporal visual cortex
Cereb. Cortex
The predictive value of changes in effective connectivity for human learning
Science
The brain's default network: anatomy, function, and relevance to disease
Ann. N.Y. Acad. Sci.
Auditory and visual word processing studied with fMRI
Hum. Brain Mapp.
Mandarin and English single word processing studied with functional magnetic resonance imaging
J. Neurosci.
Left insula activation: a marker for language attainment in bilinguals
Proc. Natl. Acad. Sci. U. S. A.
Cited by (22)
Functional organization of the ventral occipitotemporal regions for Chinese orthographic processing
2020, Journal of NeurolinguisticsRethinking the function of brain regions for reading Chinese characters in a meta-analysis of fMRI studies
2017, Journal of NeurolinguisticsCitation Excerpt :In our data, a cluster in the pMTG emerged in TC and showed more activation for TC > SC. The source foci of this cluster were from comparisons of semantic > perceptual tasks, such as meaning relatedness > perceptual judgment (Chou et al., 2009), and synonyms > pseudo-characters judgment (Chan, Tang, & Tang et al., 2009). In these comparisons, the pMTG was mainly recruited in semantic tasks, indicating lexical-semantic processing.
A meta-analysis of fMRI studies on Chinese orthographic, phonological, and semantic processing
2012, NeuroImageCitation Excerpt :Orthographic processing in Chinese is considered to involve visuospatial analysis of characters and application of a set of orthographic rules (e.g., radical position, orthographic legality) to help with character identification. Tasks in this category included orthographic judgment (whether a visually presented item was orthographically legal or illegal, e.g., (Dong et al., 2005)), stroke analysis (counting the number of a designated stroke within each presented character, e.g., (Chan et al., 2007)), orthographic search (whether the character contained a designated radical component, e.g., (Ding et al., 2003)), physical identical judgment (whether two characters or pseudo-characters presented in one trial were identical, e.g., (Chan et al., 2009; J. Liu et al., 2009; Wang et al., 2011; Zhao et al., 2010)), and font size judgment (whether the size of stimuli was small or large, e.g., (Liu et al., 2008)). The phonological task category comprised of fourteen contrasts extracted from eleven studies with a total of 233 foci.
A review and synthesis of the first 20years of PET and fMRI studies of heard speech, spoken language and reading
2012, NeuroImageCitation Excerpt :As previously documented, activation for processing word and word-like stimuli that have access to learnt abstract visual form representations was observed relative to unfamiliar nonword stimuli matched for visual complexity (Liu et al., 2008), irrespective of the hemifield of presentation (Woodhead et al., 2011a) and the physical form that the words were presented in Qiao et al. (2010) and Kronbichler et al. (2009). More details of the perceptual feature-to-whole word gradient along the posterior–anterior axis of the left occipitotemporal cortex were described for both alphabetic texts (Vinckier et al., 2007; Brem et al., 2010; Nosarti et al., 2010; Seghier and Price, 2011; Woollams et al., 2011) and Chinese/Korean texts (Chan et al., 2009). Several studies also replicated prior observations that there was remarkable similarity in the response to visual form processing of letters, words and objects (Eddy et al., 2007; Wright et al., 2008; Turkeltaub et al., 2008; Burgund et al., 2009; Kherif et al., 2011; Shinkareva et al., 2011).
Seeing Chinese characters in action: An fMRI study of the perception of writing sequences
2011, Brain and LanguageCitation Excerpt :The peak of this region lies at about Talairach coordinates [−42 −57 −12], but it extends in the anteroposterior direction across at least 2 cm (Vinckier et al., 2007). VWFA is activated for different writing systems, including Chinese (Chan et al., 2009; Fu, Chen, Smith, Iversen, & Matthews, 2002; Kuo et al., 2004; Liu et al., 2008; but see Xue, Chen, Jin, & Dong, 2006; Xue & Poldrack, 2007 which found that different aspects of the language experience, such as visual familiarity, phonology and semantics, actually all have important but different impacts on the neural activation in VWFA during language learning). It is not clear from these studies, however, whether the dynamic information associated with Chinese characters, such as the writing sequence, is encoded in VWFA, as the stimuli used were all static configurations of characters.