The occipital face area is causally involved in identity-related visual-semantic associations

Faces are processed in regions along the ventral object vision pathway (Ungerleider and Haxby 1994). However, face processing deviates from object or body processing, as faces also carry conceptual knowledge of the person (Lambon Ralph et al. 2009; Pitcher et al. 2009). The recognition of a person involves the process of identifying the face as well as learning and recalling the associated person-related knowledge, such as a name, associated and relevant places or the occupation a person. The first influential cognitive model of face recognition by Bruce and Young (1986) considered the two functional components of ‘recognizing a face’ and that of ‘encoding the relevant identity-specific semantic information’ to be inseparable. Addressing the criticism that these models assumed the existence of a single “face recognition unit” and were not based on neurophysiological data, Haxby et al. (2000) further developed the model into a neuronally motivated, two-stage network model. The core of this model, comprising the fusiform (FFA) and occipital face areas (OFA) as well as the superior temporal sulcus (STS), is involved in the early visual processing of faces. The extended system, on the other hand, including the anterior temporal pole (ATL), the amygdala, parietal as well as frontal areas (Gobbini and Haxby 2007; Rapcsak 2019), is active in higher-level cognitive processing such as semantic knowledge, emotional and motivational evaluation as well as personality traits (for a review see Ramon and Gobbini 2017).

Previously, the face-processing network was seen as a strictly hierarchical system organized in a feedforward manner (Haxby et al. 2000; Ishai 2008; Pitcher et al. 2011; Zhen et al. 2013). In these models, it was assumed that information flows from the early visual cortex toward OFA, where face detection occurs and then on to FFA for individual face recognition and to the ATL, amygdala and STS, where more complex processing, such as the identification of individual persons takes place.

In the following years, all the above-mentioned areas were studied in detail for their particular functions, often by using transcranial magnetic stimulation (TMS) where applicable (Ambrus et al. 2017a, 2019; Kadosh et al. 2011; Koivisto et al. 2017; Ranieri et al. 2015).

The highest-level unimodal area, involved in face perception, prior to memory and emotional processing is presumed to be the ATL, also called the anterior face patch (aFP; Harry et al. 2016; Rajimehr et al. 2009). Several studies found that the ATL is of importance in the identification of a person (Heide et al. 2013; Tsao et al. 2008; Tsukiura et al. 2010). Accordingly, recent TMS studies showed that stimulating the right ATL affects the feeling of familiarity for famous faces (Ranieri et al. 2015). It has been suggested that the ATL connects the visual representation of a face to the corresponding semantic knowledge one possesses about the given person (Chiou and Lambon Ralph 2018; Rice et al. 2018; Tsukiura et al. 2010). Also, hierarchically lower face-processing areas, such as the FFA, were associated with face recognition processes in the past. For instance, the FFA shows habituation to different images of the same individuals (Gauthier et al. 2000), while the OFA does not (Rotshtein et al. 2005). More recently, Axelrod and Yovel (2015) have found significant decoding of famous faces in the FFA, using multivariate pattern analysis techniques as well. These results altogether suggest that individual faces are already processed differentially at the level of the FFA and newer theories assume the FFA to be an intermediate stage of face processing. Until recently, it was believed that this is not the case for the OFA, which area is typically considered as an early, structural face-processing stage (Pitcher et al. 2007; but see Rossion (2014) for a different conclusion). However, current results suggest that the OFA is also involved in processes related to face recognition. For example, if the right OFA (rOFA) is lesioned, patients show impaired face discrimination capacities (Dricot et al. 2008; Schiltz et al. 2006). Furthermore, repetitive double-pulse TMS of the rOFA disrupts the discrimination of face parts (Pitcher et al. 2007, 2008) as well as the integration of identity and expression (Kadosh et al. 2011). Zhen et al. (2013) also argue that the OFA is involved in face identification and has a pivotal role in the face-processing network. In accordance, recent evidence suggests that the rOFA is important to identity-specific processing and to the learning of novel faces, as TMS over the rOFA (1) eliminates the effects of training in a face-matching task (Ambrus et al. 2017a), (2) reduces the observed face priming effects (Ambrus et al. 2017b) and (3) impairs discrimination of unfamiliar faces (Pitcher et al. 2008, 2009). Moreover, the participation of the OFA in processing identity-specific information is not limited to visual facial information. Amado et al. (2018) showed that the recognition of a familiar face is facilitated by the prior presentation of its name and this cross-domain priming effect is related to a reduced BOLD signal in bilateral FFA and OFA. Later, it has been demonstrated that the TMS of rOFA reduces these cross-domain priming effects, showing that the area plays a causal role in the association of semantic and visual information (Ambrus et al. 2019).

Altogether, these results imply that the OFA is more than a simple low-level, structural encoding area (Haxby et al. 2000) and that it is important to identity-specific visual and semantic processing. Traditionally, assessing personal information of a face is described as a serial process, starting with the activation of face-related information and then assessing person- and finally name-related semantic information (Tsukiura et al. 2010). Additionally, sub-networks that correspond to the recognition of an individual’s face and the retrieval of semantic knowledge are considered to be relatively independent processes (Zhen et al. 2013). Besides foundational features of the network that suggest a clearly serial processing, such as the continuously growing receptive field size along the hierarchy (Grill-Spector et al. 2017), a strictly hierarchical model of visual processing is increasingly put into question (Atkinson and Adolphs 2011). Indeed, a more complex pattern of connectivity among areas of the face network has been uncovered recently. For example, the OFA is in close exchange with the FFA via feedback connections (Gschwind et al. 2012; Jiang et al. 2011; Rossion et al. 2003). In line with this result, Kadosh et al. (2011) suggest that the OFA receives re-entrant feedback information at a mid-latency processing stage, around 170 ms. Hence, it seems that the neural response to familiar faces in the ventral occipito-temporal cortex (including the inferior occipital gyrus) reflects feedback from other face-sensitive areas as well (Gobbini et al. 2004). Furthermore, the existence of functional top-down connections from the ATL to the ventral temporal cortex has been demonstrated in object recognition (Chiou and Lambon Ralph 2016) and robust structural connectivity is reported between ATL and FFA as well as OFA (Pyles et al. 2013). Particularly, when top-down guided processes are involved, such as mnemonic functions, attention or predictions, or in this case getting to know an originally unfamiliar face via various resources, the reverse hierarchy of perceptual processes is proposed to gain more weight (Ahissar and Hochstein 2004). In summary, it seems that the face-processing neural network is far more interactive than originally thought and it has a strong interplay between the involved neural regions which may also depend on non-visual parameters, such as the task (Atkinson and Adolphs 2011; Barragan-Jason et al. 2013; Kadipasaoglu et al. 2017).

The current experiments further investigate the role of bilateral OFA in the acquisition of identity-specific semantic information. To this end, we adopted a learning paradigm where basic semantic information (the name and job of a person) was associated with a face. Importantly, TMS was applied to either the left or right OFA during the encoding phase and the participants’ performance was measured in a subsequent retrieval phase.

Face processing is a bilateral process despite its clear right hemispherical (RH) dominance (Haxby et al. 2000; Kanwisher et al. 1997). This RH dominance is evident in larger activated areas and larger face-selective event-related potential components on the right as compared to the left side (Bentin et al. 1996; Bukowski et al. 2013). This lateralization holds for faces in general (Gao et al. 2018; Rossion et al. 2012) as well as specifically for familiar face processing (Wiese et al. 2019; Wuttke and Schweinberger 2019). Interestingly, the lateralization is more evident, and it is connected to more specific functions at higher processing steps. For example, the right ATL is dominant for the processing of familiar faces, while the left ATL is more involved in the encoding of familiar names (Chiou and Lambon Ralph 2018; Ranieri et al. 2015; Rice et al. 2018). Therefore, to study the differential role of the two hemispheres in the acquisition of identity-specific semantic information, we stimulated the OFA of both hemispheres in separate experimental strands.

The major results of the current study can be summarized as follows. (1) Stimulating the OFA during the encoding of face-related semantic information reduces the correct recall of such associations for job titles, suggesting a causal role of the OFA in the association of familiar faces with related semantic information. (2) Contrary to prior findings, we did not observe hemispherical differences in this effect, suggesting a similar role of the left and right OFA in the formation of such associations.

The OFA is causally involved in semantic associations to familiar faces.

Experimental familiarization of unfamiliar faces can be performed by making fictional biographical, episodic, social or affective information available to the observers during an encoding or learning period. It is believed that rich experience with the identities facilitates the creation of a robust and long-lasting representation, modeling everyday familiarity better and facilitating the future identity recall of recognition memory. We applied TMS stimulation to the OFA during the encoding phase and this affected the subsequent recall of associated job titles with originally unfamiliar faces in both hemispheres negatively Therefore, it is likely that the OFA plays a role in the initial encoding of sensory and biographical information associations. Whether it also plays a role in other processes of recognition, such as information retrieval, is unclear and will require specifically targeting experiments. Nonetheless, our results suggest a causal involvement of OFA in the acquisition of identity-related semantic information. This conclusion is in line with prior studies showing that the OFA plays a causal role in the acquisition of face identity information as well as in the creation of cross-domain identity representation (Amado et al. 2018; Ambrus et al. 2019). Consequently, our results further support the idea that the OFA, presumably together with the other face-sensitive areas within the core network, plays a direct role in higher face-processing steps. This conclusion is consistent with prior studies showing that the OFA is involved in the formation of identity-specific memory traces as well as in face identity acquisition (). Therefore, it seems that the OFA is part of the network responsible for the identity-specific representation of a familiar face and that it also plays a role in the association of semantic information to the face of a person. Consequently, we suggest that the role of OFA and its interaction with downstream face-specific regions, such as the FFA and ATL, have to be re-evaluated in future studies. This conclusion is in line with previous lesion studies, which found normal FFA activations for faces in the absence of OFA (Rossion 2014), suggesting that the encoding of faces in the FFA is not directly dependent on its input from the OFA. This also argues against a strictly serial information processing in the core face network. Rather, it indicates that the OFA and FFA are either at similar hierarchical levels or that the FFA might even be activated independently of and earlier than the OFA (Jiang et al. 2011; Rossion 2014).

The reversed temporal course of activations is supported by BOLD latency mapping studies, which revealed somewhat earlier responses to faces in the FFA (2.02 s) than in the OFA (2.43 s; Jiang et al. 2011; Rossion et al. 2012). Note, however that there is no general agreement about this interpretation in the literature. For example, Yovel (2016) measured correlations between fMRI and EEG data and found an earlier activation onset in the OFA (104–124 ms) as compared to FFA (136–184 ms). However, this must not necessarily lead to the conclusion that the OFA is situated at a hierarchically lower processing level than the FFA. It rather suggests that the OFA is crucial to time-critical tasks, such as the discrimination of a familiar face from the face of a potentially dangerous person. Very early recognition of familiar faces is in line with recent EEG studies, which reported face identification already within 160 ms post-stimulus onset (Vida et al. 2017; Zimmermann et al. 2019). These findings, among others, contradict the simple hierarchical feedforward models of face perception (Atkinson and Adolphs 2011; Haxby et al. 2000), as these models consider OFA as the entry point of the network, responsible for the early perception of facial features (Guntupalli et al. 2017; Haxby et al. 2000).

Studies with patients having focal OFA lesions described an impairment in recognizing famous or personally familiar faces, but reported neither problems in object detection nor, more importantly, in face detection per se (Rossion 2014).These symptoms hint at the processing of general face detection in FFA and individual face recognition in OFA. On the other hand, the above-described lesion studies did not find face identity adaptation effects in the FFA (i.e., they found similar BOLD signals to identical and different faces). This suggests that the FFA nonetheless requires input from the OFA regarding the identity of the faces for successful individual face discrimination (Rossion 2014).

Altogether, it seems that the OFA, in addition to undeniably being involved in the low-level structural encoding of faces (Rotshtein et al. 2005), is likely to be involved in higher-level face-processing steps as well. This, in turn, casts doubt on the validity of the dominantly serial models of face processing (Bruce and Young 1986; Haxby et al. 2000) and suggests that the encoding behind the recognition of a familiar face is not limited to higher face-processing regions, but also includes the entire occipito-temporal network via recurrent feedback connections. Current models of face processing present a rather complex face-processing network, with multiple forward and feedback connections between each of its nodes (Duchaine and Yovel 2015). Image-independent face processing and semantic information are in close interplay across this network, explaining why semantic information is so effective when we get to know a previously unfamiliar person (Schwartz and Yovel 2016). However, in these models the OFA is still a part of the ‘early’ face-processing network and is involved in view-specific and part-based face processing (Duchaine and Yovel 2015).

The model of ‘non-hierarchical network of cortical face-selective areas’ of Rossion (2014), however, attributes the OFA a more sophisticated role than accepted by most. The author proposes a reformulated organization for early face recognition with input coming from early visual areas to the OFA, but also directly to the FFA, where face detection occurs. The connections between the FFA and the OFA have been so far always classified as feedback connections (Gschwind et al. 2012; Rossion et al. 2003). The reverse hierarchical neural model conversely describes an initial input from the FFA to the OFA, not via feedback, but rather via feedforward connections. This interpretation is supported by Kadosh et al. (2011), who report that the OFA receives information at a mid-latency processing stage, around 170 ms. Downstream processing steps refine the face representation further, through re-entrant feedback connections between higher-level areas, like the ATL, and ‘lower’-level visual areas. Indeed, Pyles et al. (2013) found strong connections between ATL and FFA as well as between the ATL and OFA. The densest connections were found, however, between the posterior and medial FFA and OFA. This reverse hierarchical neural model of individual face perception suggests the existence of a network with many re-entrant connections (for a detailed overview of person identification models, see Barton and Corrow (2016)).

Why the OFA is located at a more posterior position (compared to the FFA), while being involved in ‘higher’ face-processing steps than the FFA can be explained by the shared characteristics with the early, retinotopically organized occipital areas. Accordingly, the OFA shows robust position sensitivity (Guntupalli et al. 2017; Xu and Biederman 2010) as well as relatively small receptive fields (Grill‐Spector et al. 1998), and TMS over the right OFA impairs face shape discrimination (Freiwald et al. 2016). We know that differences in particular face features (such as the distance between eyes, mouth, nose, etc., or the size of facial parts) contribute to the recognition of individual faces. Hence, the posterior location of OFA is not due to its role in early and low-level face-processing steps, rather it suits best its complex function in recognizing facial identity. On the one hand, this hypothesis fits well with the study of Rotshtein et al. (2005), reporting that the inferior occipital gyrus (IOG) shows sensitivity to physical rather than to identity changes. On the other hand, it is also in line with studies showing face identity adaptation effects in OFA (Ewbank and Andrews 2008; Gauthier et al. 2000). Our current findings are in line with these studies, suggesting the causal role of the OFA in the association of semantic information to familiar faces.