Cross-decoding supramodal information in the human brain

Perceptual decision making is the cognitive process wherein the brain classifies stimuli into abstract categories for more efficient downstream processing. A system that, during categorization, can process information regardless of the information’s original sensory modality (i.e., a supramodal system) would have a substantial advantage over a system with dedicated processes for specific sensory modalities. While many studies have probed decision processes through the lens of one sensory modality, it remains unclear whether there are such supramodal brain areas that can flexibly process task-relevant information regardless of the original “format” of the information. To investigate supramodality, one must ensure that supramodal information exists somewhere within the functional architecture by rendering information from multiple sensory systems necessary but insufficient for categorization. To this aim, we tasked participants with categorizing auditory and tactile frequency-modulated sweeps according to learned, supramodal categories in a delayed match-to-category paradigm while we measured their blood-oxygen-level dependent signal with functional MRI. To detect supramodal information, we implemented a set of cross-modality pattern classification analyses, which demonstrated that the left caudate nucleus encodes category-level information but not stimulus-specific information (such as spatial directions and stimulus modalities), while the right inferior frontal gyrus, showing the opposite pattern, encodes stimulus-specific information but not category-level information. Given our paradigm, these results reveal abstract representations in the brain that are independent of motor, semantic, and sensory-specific processing, instead reflecting supramodal, categorical information, which points to the caudate nucleus as a locus of cognitive processes involved in complex behavior.